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SECTION  1 


INTRODUCTION  AND  SUMMARY 


The  development  and  maintenance  of  test  programs  have  become  one  of 
the  main  cost  factors  in  Air  Force  weapons  systems.  One  of  the  major 
components  of  these  development  costs  is  Verification  and  Validation 
(V&V).  This  report  addresses  the  use  of  a  Unit  Under  Test  (UUT)  simulator 
as  a  potential  solution  in  reducing  V&V  costs  and  in  increasing  the  effective¬ 
ness  of  the  V&V. 

The  typical  approach  during  test  program  V&V  is  to  execute  the  manually- 
generated  test  programs  on  the  Automatic  Test  Equipment  (ATE),  using 
a  real  UUT  to  verify  that  the  programs  operate  as  intended.  The  following 
are  some  typical  problems  which  are  encountered: 

•  UUT  unavailability  during  the  V&V  phase 

•  Delay  of  test  program  verification  because  the  spare  UUTs  being 
used  are  pulled  back  for  higher  priority  needs 

•  Delays  because  of  UUT  damage  due  to  errors  in  the  test 
programs 

•  Uncertainty  of  test-program  coverage  because  fault  paths 
cannot  be  forced  in  the  UUT  being  used  for  V&V. 

It  was  determined  by  the  Air  Force  that  one  potential  solution  to  the 
above  problems  is  to  use  a  UUT  simulator  rather  than  a  real  UUT  during 


as  much  of  the  V&V  activity  as  possible.  To  assess  the  merit  of  this 
approach,  the  Air  Force  initiated  the  UUT  Simulator  Feasibility  Investigation 
Study  Program.  This  report  covers  the  findings  in  the  performance  of  this 
program. 

The  study  approach  basically  involved  a  top-down  look  at  the  problem 
to  be  solved  and  a  bottom- up  look  at  the  techniques  and  technologies  that 
could  be  applied  to  realize  a  UUT  simulator.  During  the  first  part  of  the 
program  the  following  top-down  activities  were  performed: 

•  Requirements  were  collected  from  the  literature,  from  a 
validator  survey,  from  analyzing  Test  Requirement  Documents 
(TRDs),  and  from  analyzing  representative  ATLAS  test  programs. 

•  A  baseline  set  of  requirements  was  established. 

•  A  set  of  enhancement  features  was  defined  and  prioritized. 

•  UUT  simulator  usage  scenarios  were  postulated. 

•  Potential  Interface  Adapter  (IA)  verification  usage  was  defined. 

The  specifics  of  the  above  are  covered  in  Sections  2  and  3  of  this  report. 

As  a  part  of  the  bottom-up  activities,  the  following  were  performed: 

•  The  literature  was  searched  for  potentially  applicable 
simulation  techniques. 

•  Extensions  of  the  techniques  and  specific  sets  of  UUT  simulator 
solutions  were  postulated. 

•  Trade-off  information  was  derived  for  the  classes  of  UUT 
simulators. 
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Based  on  this  information,  recommended  solutions  were  derived. 


•  Additional  uses  for  a  UUT  simulator  aside  from  supporting  V&V 
were  considered. 

The  results  of  the  above  are  covered  in  Sections  4  and  5  of  this  report. 

The  following  set  of  conclusions  summarizes  the  outcome  of  the  UUT 
simulator  feasibility  study  program: 

•  The  feasibility  of  simulators  for  classes  of  UUTs  in  the  digital, 
analog,  and  hybrid  area  is  clear;  their  cost-effectiveness  in 
V&V  must  still  be  validated. 

•  To  be  most  effective  the  UUT  simulator  development  must  start 
early  in  the  development  cycle  of  a  Line  Replaceable  Unit  (LRU), 
and  the  simulator  must  be  based  on  the  UUT  design  or  schematics. 
From  Modular  Automatic  Test  Equipment  (MATE)  program  inputs, 
it  was  found  that  the  UUT  simulator  should  be  oriented  more 
toward  ensuring  that  the  V&V  provides  better  checks  on  the  TRD 
and  on  the  test  programs  than  toward  minimizing  the  cost  of 
developing  test  programs. 

•  The  UUT  simulator  should  be  used  as  an  orthogonal  check  on 
the  validity  and  completeness  of  the  testing  as  embodied  in  the 
TRD  and  in  the  test  programs.  Assessment  of  fault  coverage 
completeness  to  minimize  LRU  down-time  when  in  the  field  is  a 
key  payoff  factor. 

•  The  UUT  simulator  should  be  based  on  multiple  levels  of 
simulation  and  on  either  a  software-only  or  an  instrument  bus 
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interface  with  the  ATE  system  on  which  the  test  programs  are 
executed.  Solutions  based  on  analog  signal  interfaces  are  not 
recommended. 

•  In  using  the  UUT  simulator  as  recommended  above,  the  Test 
Package  Set  (TPS)  development  costs  may  actually  increase.  How¬ 
ever,  the  resulting  increase  in  test  program  quality  (and 
consequently  improved  UUT  uptime)  should  more  than  offset  this 
increase. 

•  A  UUT  simulator  is  not  a  total  or  the  only  solution  in  supporting 
V&V  and  in  decreasing  test  program  development  and  maintenance 
costs.  A  real  UUT  must  still  be  used  during  a  part  of  the  validation. 
Automatic  test  program  generation,  if  realizable  and  cost-effective, 
reduces  the  need  for  a  UUT  simulator. 

•  Test  program  developers  and  validators  must  be  convinced  of  the 
merits  of  a  UUT  simulator.  A  demonstration  program  addressing 
further  issues  and  supporting  the  validation  of  the  UUT  simulator 
concept  is  necessary  before  large  scale  usage  of  UUT  simulators. 

•  Additional  UUT  simulator  cost-effectiveness  issues  must  be 
resolved.  It  is  not  clear  yet  whether  it  is  most  cost-effective 
to  develop  a  generic  UUT  simulator  system,  whether  families 
of  such  systems  should  be  developed,  or  whether  it  will  suffice 
to  simply  develop  a  set  of  guidelines  to  support  the  construction 
of  unique  UUT  simulators  based  on  specific  weapon  program 
requirements. 
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SECTION  2 


PROBLEM  DEFINITION 


This  section  presents  a  more  detailed  picture  of  the  problem  that  the  UUT 
simulator  is  addressing.  Specific  examples  are  used  to  demonstrate  to 
the  reader  the  reality  of  the  problem.  Experiences  with  a  very  low  fidelity 
form  of  simulator  are  also  described. 

VALIDATION  AND  VERIFICATION  (V&V) 

Verification  and  validation  of  UUT  test  programs  involve  a  complex  set  of 
disciplines.  When  the  validator  encounters  a  problem  implementing  a  test, 
he  must  discriminate  where  the  problem  originates: 

•  Is  it  the  test  program  software? 

•  Is  the  test  program  stepping  outside  the  ATE  capabilities? 

•  Is  the  ATE  itself  at  fault?  If  so,  is  it  by  design  deficiency  or 
malfunction? 

•  Is  the  interface  test  adapter  mating  the  UUT  to  the  ATE  at  fault? 

If  so,  is  it  be  design  deficiency  or  malfunction? 

•  Is  the  UUT  failing?  If  so,  how  can  this  be  verified? 

•  Is  the  Test  Requirement  Document  (TRD)  for  the  UUT  accurate, 
up-to-date,  and  correct? 
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Figure  1  depicts  one  version  of  an  ATLAS  test  program  development 
process.  Though  greatly  affected  by  prior  elements,  V&V  begins  at  the 
step  "Validate  Program  with  Adapter  and  UUT"  and  continues  beyond 
acceptance  of  the  first  Test  Package  Set  (TPS).  The  reason  V&V  actually 
continues  beyond  "sell  off"  is  perceivable  when  considering  multiple  test 
systems  deployment.  Each  ATE  system  must  "play"  with  each  of  its  TPSs 
to  be  operational.  Experience  at  Warner  Robins  Air  Logistic  Command 
(WRALC)  has  shown  that  because  of  UUT  variability,  there  is  only  25 
percent  confidence  that  a  TPS,  when  validated  with  only  one  UUT,  will 
work  with  all  UUTs.  This  confidence  jumps  to  75  percent  for  a  TPS 
validated  against  two  UUTs  and  90  percent  for  three  UUTs. 

Typically,  however,  V&V  is  thought  of  as  just  through  sell  off  of  the  first 
TPS.  When  used  in  this  context,  V&V  usually  accounts  for  about  40 
percent  of  the  TPS  development  process. 

PROBLEM  IS  REAL  AND  COSTLY 

To  characterize  the  problem,  let  us  look  at  an  example  derived  from 
Honeywell  experience.  As  an  ATE  supplier  of  the  F-15  ADTS  and  F-15 
TITE  systems,  Honeywell  has  total  ATE  and  TPS  development,  integration, 
and  V&V  responsibility.  This  V&V  activity  has  produced  over  500  Avionics 
LRU  and  module  TPSs  during  the  last  five  years.  Because  each  TPS 
effort  involved  usage  of  at  least  one  UUT,  at  a  few  thousand  dollars  value 
each,  many  millions  of  dollars  worth  of  UUTs  have  been  tied  up  during 
this  V&V  effort.  In  the  case  of  LRUs,  the  problem  can  sometimes  be  more 
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Figure  1.  ATLAS  Test  Program  Development  Process 
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noticeable  because  the  V&V  cycle  is  usually  longer  and  the  LRU  cycle  is 
more  costly.  For  example,  on  F-15  TITE,  LRU  No.  3  (Countermeasures 
Computer)  required  availability  of  three  LRUs  for  one  and  one-half  years 
during  V&V. 

UUT  AVAILABILITY  PROBLEM  EXAMPLES 

At  Honeywell  we  have  found  that  more  typically  LRU  V&V  requires  an 
availability  of  one  and  one-half  LRUs.  The  extra  one-half  LRU  would 
generally  be  for  card/plug-in  module  replacements  for  trouble-shooting 
and  repair  of  the  LRU  in  V&V.  While  this  is  possible  with  LRUs,  it  is 
not  always  practical,  and  it  will  not  suit  module  level  V&V  at  all. 

These  practical  limitations  on  V&V  activities  were  very  clearly  illustrated 
in  the  minutes  of  two  recent  F-15  TITE  Technical  Meetings: 

•  "There  is  a  shortage  of  Band  1  UUTs.  There  is  a  total  of  four 
Band  1  UUTs;  three  of  these  four  UUTs  are  defective.  The 
only  good  UUT  is  being  used  in  RF  amplifier  testing.  Two  of  the 
other  UUTs  are  being  returned  to  St.  Charles  for  repair.  The 
other  defective  UUT  will  be  kept  at  the  site  (V&V)  for  limited  use 
during  validation.  The  diodes  required  for  repair  of  these  UUTs 
have  arrived  from  Northrop.  " 

•  "Verification  of  ATE  Stations  10  through  15  at  Honeywell, 
Minneapolis,  presents  no  problems  for  the  ICS  UUTs;  however, 
TITE  does  not  have  available  enough  EWWS  and  RWR  UUTs  for 
verification  at  Honeywell,  Minneapolis,  while  maintaining  UUTs 
at  the  St.  Louis  site  for  sustaining  work.  " 


UUT  DAMAGE/REFURBISHMENT  PROBLEMS 


Going  beyond  availability  of  good  UUTs,  V&V  faces  the  inevitable  damage 
of  a  UUT  during  V&V.  Typical  causes  for  such  UUT  damage  are  as 
follows: 

•  Oscilloscope  probe  damage  to  Avionics  module  protective 
coating  of  components 

•  Inadvertent  electrical  damage  to  the  UUT  by  grounding  with 
probes 

•  Incorrect  application  of  UUT  power 

While  the  first  cause  may  be  understandable  in  a  validation  environment 
and  not  too  expensive  to  repair,  the  second  and  third  causes  can  be  very 
costly  because  damage  to  both  the  UUT  and  ATE  can  be  extensive.  But 
even  if  damage  is  minimal,  a  defective  (damaged)  UUT  will  usually  shut 
down  V&V  effort  because  a  second  UUT  is  not  available.  Honeywell 
experience  on  F-15  shows  the  average  time  to  repair  a  UUT  is  one  to 
two  weeks,  and  a  minimum  of  3  days  is  required.  This  discontinuity 
in  the  V&V  effort  has  a  very  adverse  effect  on  cost  and  schedules  for 
ATE  projects. 

ELEMENTS  OF  A  UUT  SIMULATOR 

Classes  of  UUTs 


The  typical  requirements  for  UUT  availability  in  a  TPS  development 
project  are  portrayed  in  Figure  2  (solid  line).  The  dashed  lines  in  this 


NUMBER  OF 
UUT* NEEDED 
FOR  V&V 


YEARS  AFTER  ATE  CONTRACT  AWARD 

-  Typical  requirements  for  UUT  availability  in  a  TPS  development  project 

- Goal  of  reducing  need  for  UUT 

Figure  2.  UUT  Requirements  at  V&V  Efforts 

figure  illustrate  the  goal  of  driving  down  the  need  for  UUTs  and  reducing  the 
time  they  are  involved  in  V&V.  If  a  universal  UUT  simulator  were  available 
that  could  stand  up  to  V&V  in  place  of  any  UUT,  this  could  indeed  drive  the 
dashed  line  near  zero.  This  problem  is  bounded  somewhat  by  requiring 
coverage  of  only  analog,  digital,  and  hybrid  UUTs.  However,  all  classes 
of  UUTs  are  encountered  in  TPS  development  and  as  such  these  "classes"-- 

•  Analog 

•  Digital 

•  Hybrid 

•  High  Power 

•  RF 
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•  Microwave 


•  Video 

•  Pneumatic 

•  Inertial 

•  Communication 

should  also  be  considered  as  worthy  targets  of  the  solution. 

The  ATE  Itself 

The  ATE  where  TPS  development  is  conducted  should  also  be  considered 
an  element  in  any  simulation  approach.  Modularization  of  ATE  promised 
by  the  MATE  program  could  have  a  dramatic  impact  on  a  UUT  simulator 
solution.  If  interfaces  in  MATE  are  standard  and  accessible,  a  simulator 
may  be  able  to  intercept  native  ATE  functions  rather  than  be  a  slave  to  the 
general  ATE  I/O  ports. 

Perhaps,  though,  the  ideal  simulator  would  not  concern  itself  with  the  ATE, 
but  rather  conduct  itself  as  the  UUT  would— responding  to  ATE  stimuli  with 
an  expected  UUT  response.  In  this  way,  the  UUT  simulator  could  be  used 
on  any  ATE,  not  just  those  with  MATE-defined  interfaces.  Besides,  it  may 
not  be  practical  to  allow  a  V&V  effort  to  have  access  to  internal  ATE 
functions. 


Limited  Data 


Any  UUT  simulator  must  also  address  the  problems  of  attaining  information 
about  the  UUT.  UUT  Test  Requirements  Documents  (TRD)  misinformation 
has  been  proven,  time  and  time  again,  to  be  a  back  breaker  of  TPS  develop¬ 
ment  programs.  Acquiring  information  necessary  for  UUT  simulation  could 
be  an  equally  frustrating  experience.  For  a  simulator  to  conduct  itself  as 
the  UUT  would  perform,  it  may  be  necessary  to  obtain  UUT  design  informa¬ 
tion  as  well  as  UUT  performance  information  as  contained  in  the  TRD.  This 
may  be  another  practical  limit  to  simulation,  considering  the  consequences 
of  vendor  proprietary  design  information. 

Consideration  should  be  given  to  approaches  to  simulation  that  minimize 
the  need  for  design  information  about  the  UUT.  Perhaps  the  simulator 
should  concern  itself  with  only  the  UUT  I/O  interfaces — the  signal  types, 
ranges  of  stimulus  and  response  constraints,  data  envelop  formats,  etc. 

Such  information,  because  it  may  be  more  "eadily  attainable  and  be  fairly 
correct  (because  a  companion  LRU  or  module  depends  on  this  interface), 
may  be  a  viable  data  base  for  construction  of  a  UUT  simulator. 

Can  the  UUT  Be  Eliminated? 


Unless  one  narrows  and  descopes  the  classes  of  UUTs  and  the  types  of 
tests  validated  on  these  UUTs  (i.  e. ,  simple,  moderate,  or  complex 
synchronized  tests),  it  would  be  difficult,  if  not  impossible,  to  totally 
eliminate  the  UUT  from  V&V.  A  UUT  simulator  approach  that  could 
eliminate  the  need  for  a  UUT  during  99  percent  of  the  V&V  effort,  but  was 
effective  only  for  digital  card  UUTs,  might  be  easy  to  achieve.  But 


another  approach,  which  would  eliminate  the  need  for  a  UUT  80  to  90 
percent  of  the  time  and  which  would  cover  a  broader  class  of  digital,  analog, 
and  hybrid  LRUs  or  modules  may,  indeed,  provide  a  better  ratio  of 
performance  to  development  cost. 

A  relatively  simple  but  effective  simulation  approach  was  recently 
developed  by  Honeywell  for  the  F-16  Depot  TPS  development.  For  F-16 
Depot,  Honeywell  was  chosen  to  supply  analog  test  stations.  General 
Dynamics/Fort  Worth  (GD/FW)  had  TPS  development  responsibility. 

Because  of  schedule  constraints,  GD/FW  needed  a  mechanism  that  would 
allow  them  to  begin  TPS  development  prior  to  delivery  of  analog  test 
stations.  As  a  result,  Honeywell  delivered  the  F-16  Depot  Mini-Disc 
Expanded  Memory  System  (UG26000GD01 )  in  August  1978.  As  in  many 
ATE  program  development  stations,  this  system  allowed  test  program 
•generation  to  proceed  in  a  very  effective  manner.  But  bec'ause*  this  system, 
called  Mini-DEMS,  was  implemented  with  a  set  of  computer  and  peripheral 
devices  identical  to  the  analog  test  station,  TPS  V&V,  as  well  as  test 
system  training  capabilities,  were  added.  This  was  accomplished  by 
adding  software  and  a  circuit  card  to  the  computer  I/O  bus  extension 
that  would  act  as  a  null  interface,  "simulating"  any  analog  test  station 
ATE  instrumentation.  The  Mini-DEMS  has  allowed  GD/FW  to  actually 
execute  the  ATLAS  test  programs  they  develop.  This  Mini-DEMS  has 
allowed  the  V&V  process  to  proceed,  eliminating  not  only  the  need  for  a 
UUT,  but  also  the  ATE  itself  and  the  interface  test  adapter.  In  addition, 
because  the  Mini-DEMS  reacts  just  like  the  analog  test  station,  it  has 
served  as  a  very  effective  ATE  operation  trainer  also.  Two  Mini-DEMS 
were  delivered  to  GD/FW,  one  in  August  1978,  the  other  in  September  1978 
(one  will  be  delivered  to  USAF  later). 


TPS  development  and  V&V  have  proceeded  on  a  multishift  basis  at  GD/FW 
since  this  delivery.  Those  test  programs  that  had  completed  V&V  testing 
on  a  Mini-DEMS  were  ready  for  final  integration  with  the  ATE  which 
arrived  in  January  1979.  Using  this  approach,  30  to  40  percent  of  the 
V&V  effort  was  possible  without  the  UUT. 


SECTION  3 


UUT  SIMULATOR  REQUIREMENTS 


This  chapter  contains  the  definitions  of  the  requirements  which  must  be 
satisfied  in  order  for  the  UUT  simulator  to  realize  the  projected  benefits. 
The  first  section  contains  a  brief  description  of  the  methods  used  to 
collect  the  requirements.  It  shows  that  a  comprehensive  multi-faceted 
approach  was  used  to  arrive  at  the  requirements.  Reaching  the  same 
conclusions  with  different  approaches  lends  credibility  to  the  requirements. 

The  second  section  contains  a  definition  of  the  baseline  requirements  for 
the  UUT  simulator.  This  set  of  requirements  is  fundamental  to  the 
operation  of  the  simulator  in  the  anticipated  scenarios.  If  these  require¬ 
ments  are  not  met,  we  project  that  the  expected  benefits  of  the  UUT 
simulator  will  be  lessened. 

The  third  section  contains  a  definition  of  the  UUT  simulator  enhancements, 

prioritized  according  to  their  expected  value.  The  features  described 

* 

here  are  expected  to  provide  some  payoff  if  included  in  the  simulator,  but 
they  are  "wants"  rather  than  "musts."  The  prioritization  was  performed 
on  a  basis  of  payoff  rather  than  on  a  basis  of  expected  additional  cost  of 
the  feature. 

The  remaining  sections  of  this  chapter  contain  additional  documentation 
supporting  the  baseline  requirements  and  enhancements.  The  topics 
discussed  are  requirements  information  from  the  literature,  questionnaire 
results,  and  TRD/ ATLAS  program  analyses. 
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REQUIREMENTS  COLLECTION  METHODS 


A  properly  scoped  activity  to  collect  information,  which  impacts  the 
requirements  definition,  is  the  key  first  step  in  arriving  at  a  good  UUT 
simulator  definition.  The  collection  must  be  performed  from  different 
angles  of  attack  to  ensure  a  good  composite  picture  of  what  the  problem 
is  and  what  is  needed.  This  was  accomplished  on  our  program  by  perform¬ 
ing  the  following  activities: 

•  A  Literature  Survey— A  number  of  approaches  was  used  to  search 
for  relevant  information  in  the  literature,  which  would  help  to 
clarify  the  nature  and  extent  of  the  problem  being  addressed, 
which  would  describe  relevant  experience  with  simulation 
techniques  or  other  methods  for  overcoming  the  problems, 

and  which  would  possibly  identify  solutions  that  have  already  been 
used.  During  the  literature  search  we  did  not  concentrate  as 
much  on  solutions  as  on  trying  to  identify  the  problem.  We  were 
more  interested  during  this  stage  of  the  program  in  information 
that  would  lead  to  requirements  rather  than  solutions. 

The  results  of  the  literature  survey  are  discussed  in  more  detail  in 
this  section.  These  results  and  further  observations  from  papers 
which  we  reviewed  but  did  not  document  had  a  strong  influence  on 
the  UUT  simulator  requirements  defined  in  this  chapter. 

•  A  Survey  Questionnaire-Based  on  preliminary  discussions  with 
test  program  developers  and  validators,  a  survey  questionnaire 
was  developed  and  distributed  both  within  Honeywell  and  to  outside 
experts.  The  results  from  the  responses  and  in  some  cases 


verbal  discussions  helped  to  formulate  requirements  constraints. 

By  using  the  questionnaire  we  were  able  to  find  information  which 
typically  would  not  appear  in  formal  papers  in  conferences. 

•  TRD/ ATLAS  Program  Analysis  — A  representative  set  of  test 
requirements  documents  and  ATLAS  test  programs  was  chosen 
from  the  F-15  TITE  program.  These  were  then  analyzed  for 
complexity  and  for  characteristics  which  would  influence  the 
requirements  for  a  UUT  simulator.  The  detailed  results  of  this 
analysis  are  found  in  this  section.  Again,  the  UUT  simulator 
requirements  presented  in  this  chapter  reflect  what  was  learned 
in  the  analysis. 

When  all  the  information  had  been  collected,  we  looked  at  a  number  of 

alternative  ways  of  coalescing  it  and  presenting  it.  We  finally  decided  to 

•  •  *  •  •  •  • 

use  the  form  of  categorization  of  the  requirements  as  shown  in  Table  1. 

This  method  allows  a  reasonably  orthogonal  presentation  of  the  UUT 
simulator  features  and  supports  a  structured  presentation  which  will  make 
it  easier  for  the  reader  to  keep  the  requirements  separated.  The  following 
briefly  discuss  the  categories  which  are  used. 

UUT  Simulator  Start  Time  (R.  1) 

This  requirement  has  to  do  with  the  relative  phasing  of  the  UUT  development, 
the  development  of  the  test  program  set  for  the  UUT,  and  the  development  of 
the  UUT  simulator.  In  the  ideal  case,  the  UUT  simulator  could  be  started 
when  the  UUT  is  still  in  design.  In  some  cases  the  UUT  simulator  must  be 


17 


TABLE  1.  UUT  SIMULATOR  REQUIREMENTS  CATEGORIES 


R.  1 

UUT  Simulator  Start  Time 

R.  2 

Modeling  Levels 

R.  3 

Physical  Requirements 

R.  4 

Time  Requirements 

R.  5 

V&V  Session  Functionality 

5.  1 

Initialization 

5.2 

Interactive  Simulation  Session 

5.  3 

Interactive  Session  Control 

5.4 

Performance  Constraints 

5.  5 

Simulation  Session  Preparation 

5.  6 

Post-Simulation  Evaluation 

R.  6 

Simulation  System  Control  Support 

developed  after  the  UUT  has  already  been  fully  developed.  It  is  still 
important  to  have  adequate  design  and  interface  information  to  be  able  to 
develop  an  adequate  simulator. 

Modeling  Levels  (R.  2) 

The  level  of  modeling  of  the  UUT  determines  the  level  of  fidelity  at  which 
the  UUT  simulator  mimics  the  actual  UUT.  At  one  extreme,  digital  and 
analog  UUTs  are  simulated  with  precise  circuit  models.  At  the  other 
extreme,  only  the  interfaces  which  interact  with  the  ATE  equipment  are 
sufficiently  modeled  to  produce  representative  responses.  The  two  extremes 
can  also  be  considered  analogous  to  models  used  in  electronic  maintenance 
training  simulators: 
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•  A  "simulation  based"  maintenance  trainer  is  one  in  which  the 
interaction ^s  based  on  simulation  models  of  the  device  being 
trained.  Internal  states  are  maintained  so  that  the  simulation  is 
reasonably  realistic. 

•  A  "procedural"  trainer  is  one  in  which  only  the  interface  is 
simulated  to  a  sufficient  degree  to  allow  a  student  to  follow  a 
sequence  of  steps  in  a  Technical  Order. 

The  "procedural"  form  of  trainer  corresponds  to  a  UUT  simulator  based 
on  interface  characteristics  only.  This  form  of  a  simulation  tends  to  be 
less  expensive,  but  adequancy  of  fidelity  is  an  issue. 

For  automatic  test  program  verification  there  are  additional  modeling 
requirements  beyond  the  UUT:  the  interface  adapter,  any  operator 
interfaces  (such  as  pots  or  switches),  and  the  ATE  equipment  (depending 
on  the  simulation  approach). 

Physical  Requirements  (R.  3) 

These  requirements  have  to  do  with  the  required  level  of  ruggedness  of  the 
equipment  performing  the  UUT  simulation,  the  location  of  the  equipment 
relative  to  the  ATE  to  which  the  UUT  is  connected,  and  the  fidelity  of  the 
electrical  signals  at  the  interface.  Physical  portability  is  also  a  require¬ 
ment  to  be  considered  in  this  category. 
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Time  Requirements  (R.  4) 


There  are  a  number  of  different  aspects  of  time  which  may  be  required 
by  the  UUT  simulator: 

•  Operation  speed  as  compared  to  real  time 

•  Block  allocation  time  which  may  constrain  the  user  access  to 
the  simulator 

•  Immediacy  of  access 

•  Response  time  to  activities  requested  by  the  user 

•  Familiarization  time  for  a  user  to  become  proficient  in  the 
use  of  the  simulator 

V&V  Session  Functionality  (R.  5) 


The  requirements  in  this  category  have  to  do  with  the  use  of  the  UUT 
simulator  during  a  session  in  which  an  automatic  test  program  is  being 
verified.  To  help  partition  the  requirements  into  a  more  structured 
format,  the  following  subsections  are  used: 

•  Initialization  (R.  5. 1 ) 

•  Interactive  Simulation  Session  (R.  5.  2) 

•  Interactive  Session  Control  (R.  5.  3) 

•  Performance  Constraints  (R.  5.  4) 

•  Simulation  Session  Preparation  (R.  5.  5) 

•  Post-Simulation  Evaluation  (R.  5.  6) 

Each  of  these  subsections  is  briefly  discussed  below. 
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Initialization  (R.  5. 1 )— Initialization  includes  activities  required  to  start 
the  system,  to  activate  the  desired  UUT  simulation  model,  to  start  the 
operator  interface,  and  to  activate  auxiliary  models  such  as  the  Interface 
Adapter  model  if  present. 

Interactive  Simulation  Session  (R.  5.  2)— Requirements  in  this  subsection 
are  key  in  making  the  UUT  simulator  achieve  the  expected  benefits.  The 
user  must  be  able  to  adequately  control  the  interaction  of  the  simulation 
with  the  ATE  and  the  external  environment.  Furthermore,  the  simulation 
must  provide  him  with  feedback  and  must  be  adequately  responsive  so  that 
he  uses  his  time  in  debugging  and  verifying  test  programs  efficiently.  The 
requirements  are  further  divided  into  the  following  categories: 

Input  Requirements  (R.  5.  2.  1)— All  inputs  which  must  be  sensed  by 

the  UUT  simulator  from  the  operator,  from  the  ATE,  and  from  the  simu- 
•  •  •  •  « 

lated  environment  are  included.  These  inputs  are  simulations  of  equivalent 
inputs  which  must  be  sensed  by  the  real  UUT. 

Output  Requirements  (R.  5,  2.  2)-" All  the  outputs  which  must  be 
generated  by  the  UUT  simulator  are  included.  Also  specified  in  this 
section  are  the  means  by  which  the  outputs  are  calculated. 

Simulation  Status  Outputs  (R,  5.  2.  3)— In  addition  to  the  outputs  which 
mimic  the  real  UUT  outputs,  the  UUT  simulator  can  have  additional  internal 
states  which  can  help  in  the  process  of  debugging  and  verifying  the  test 
programs.  This  section  contains  the  requirements  for  outputting  such 
information. 
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Performance  Monitoring  (R.  5.  2.  4)— During  the  process  of  simulating 
a  UUT  being  exercised  by  a  specific  test  program,  information  can  be 
collected  on  the  performance  of  the  test  program  and  on  the  performance  of 
the  UUT.  This  information  can  then  be  used  in  evaluating  how  well  the 
test  program  is  exercising  the  UUT. 

On-Line  Performance  Evaluation  (R.  5.  2.  5)— Functional  requirements 
in  this  category  determine  what  performance  information  can  be  extracted 
while  the  simulation  is  still  in  progress.  As  a  result  of  getting  such 
information,  the  user  makes  changes  in  the  test  program  or  in  the  method 
of  simulation  to  improve  the  performance  indicators  which  he  is  watching. 

Procedural  Branching  Control  (R.  5.  2.  6)— An  important  aspect  of  a  UUT 
simulator  is  the  ability  to  force  the  test  program  into  exercising  all  of  the 
test  branches.  The  requirements  determine  the  different  types  of  procedural 
control  needed. 

Patch  and  Retain  Temporary  Changes  (R.  5,  2.  7)— During  the  debug 
phase  of  test  program  development  it  is  necessary  to  allow  the  programmer 
to  make  incremental  modifications  so  that  he  can  gauge  their  effect  on  the 
simulation  and  on  how  the  test  program  responds. 

Interactive  Session  Control  (R.  5.  3)— This  set  of  requirements  characterizes 
how  the  user  can  control  the  progress  of  his  session  on  the  UUT  simulator. 
The  requirements  are  further  classified  into  the  following  categories: 
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Freeze/Run/Replay  (R.  5.  3. 1 )— Capabilities  are  specified  here  as  to 
the  extent  of  control  of  the  simulation  in  terms  of  stopping  it,  allowing  it 
to  continue,  and  restarting  it  at  a  previous  point. 

Monitoring  Level  Control  (R.  5.  3.  2)— The  requirements  in  this  section 
determine  the  amount  of  control  over  tracing  of  simulator  variables. 

Utility  Functions  (R,  5.  3.  3)— This  section  includes  requirements  for 
special  utility  functions  which  are  needed  to  support  control  of  the  simulation. 

UUT  Simulator  Messages  (R.  5.  3.  4)— The  requirements  in  this  section 
include  the  types  of  messages  which  must  be  provided  to  indicate  to  the 
user  that  the  simulator  has  encountered  error  conditions  or  that  capacities 
of  the  system  are  close  to  being  exceeded. 

Performance  Constraints  (R.  5.  4)— This  section  contains  requirements  on 
the  performance  of  the  UUT  simulator  during  a  V&V  Session.  The  two  areas 
covered  are  performance,  when  used  in  a  batch  mode,  and  responsiveness, 
when  used  in  an  on-line  mode. 


Simulation  Session  Preparation  (R,  5.  5)— Requirements  in  this  category 
are  concerned  with  the  features  in  the  UUT  simulator,  which  are  oriented 
to  helping  the  user  plan  and  prepare  for  a  simulation  session.  The 
following  are  the  subsections: 
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Function  Selection  (R,  5.  5. 1 )— The  requirements  are  included  for 
selecting  specific  cases  from  a  set  of  alternatives  available.  For  example, 
selection  of  a  specific  UUT  and  selection  of  specific  operating  modes  may 
have  to  be  supported. 

Malfunction  Selection  (R,  5.  5,  2)— The  requirements  in  this  section 
include  the  ability  to  force  the  simulator  to  mimic  specified  malfunctions 
which  will,  in  turn,  force  the  test  program  into  specific  procedures. 

Procedure  Selection  (R.  5.  5,  3)— If  response  in  the  simulation  can  be 
determined  by  procedures,  this  section  contains  requirements  on  the 
selection  and  modification  of  these  procedures. 

Simulation  Preparation/ Selection  (R.  5.  5.  4)— This  section  includes 
requirements  on  the  ability  to  prepare  specifics  of  the  simulation  such 
as  amount  of  detail  and  values  for  specific  parameters. 

Trace  Level  Selection  (R.  5.  5.  5)— The  requirements  included  determine 
the  ability  to  preset  trace  levels  and  detail  before  the  simulation  is 
started. 

Post-Simulation  Evaluation  (R,  5.  6)— The  requirements  in  this  section 
include  capabilities  of  the  system  to  support  evaluation  of  results  after 
the  simulation  has  been  completed.  The  following  are  the  subsections 
included: 
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Test  Program  Evaluation  (R.  5.  6. 1 ) 


Save  Performance  Reports  (R.  5.6.  2) 


Display  Specific  Reports  (R.  5.  6.  3) 


Simulation  System  Control  Support  (R.  6) 


Requirements  in  this  section  include  those  features  of  a  computer  system 
required  to  keep  the  system  operational  and  to  keep  the  system  updated 
with  the  latest  information  on  the  simulation  components.  The  following 
subsections  are  included: 

Initialization  (R.  6.  1 ) 


Operational  Assurance  (R.  6.  2) 


Subsystem  Switching  Control  (R.  6.  3) 


Off-Line  Activity  Control  (R.  6.  4) 


System  Instrumentation  (R.  6.  5) 


System  Recovery  (R.  6.  6) 


Configuration  Management  Support  (R.  6.  7) 


Performance  Data  Reduction  Support  (R.  6.  8) 


Load  Pro  iection/ Enhancement  Support  (R.  6.  9) 


Data  Base  Support  (R.  6.  10) 


Off-Line  Preparation/ Maintenance  (R.  6. 11) 


A  summary  of  the  UUT  simulator  requirements  results  is  presented  in 
the  matrix  in  Table  2.  The  requirements  categories  discussed  above 
appear  with  two  columns.  The  first  column  contains  an  "x"  if  the 
corresponding  requirement  is  in  die  baseline.  If  there  is  no  "x"  m  the 
first  column,  the  feature  is  classified  as  an  enhancement.  In  this  case 
there  is  a  number  from  1  to  3  in  column  2.  A  "3"  indicates  high  value 
of  the  feature  and  a  "l”  indicates  a  low  value.  The  individual  requirements 
are  further  explained  in  the  following  two  sections  of  this  chapter. 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 
REQUIREMENTS  RESULTS 


i 

I 


1.  UUT  Simulator  Start  Time 

At  UUT  Design  !  12! 

After  UUT  has  been  built  !  x  i  1 

•  mm  •  •  - 


2.  Modeling  Levels 

Circuit  Level 
Analog 

Topology  and  Simple  Models 
Topology  and  Good  Models 
Good  Models  and  Second  Order 
Coupling  Effects 

Digital 

Register  Transfer  Model 
Register  Transfer  Model  and 
Timing  Model 
Detailed  Circuit  Model 

Interface  Level 

Recognize  Stimuli  and  Generate 
Response  (from  bus) 

Recognize  Stimuli  and  Generate 
Accurate  Electrical  Signals 

Additional  External  Models 
Interface  Adapter  Model 
Topology 
Circuits 

Operator  Interface  Model 
ATE  Equipment  Model 


2 

2 

1 


2 

2 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


Modeling  Language 

English  Language  Based  I  x  I  ! 

( ATLAS-like) 

Graphical  !  !  2  ] 


Fault  Simulation  Capability 

Interface  Level 
Functional  Module  Level 
Circuit  Level 


2 

1 


3.  Physical  Requirements 


Hardware  Reliability 

Militari zed  !  ]  1 

Ruggedized  ]  I  2 

Commercial  !  x  ! 

Locat ion 

Adjacent  ATE  !  !  3 

Remote  and  Real  Time  Access  !  !  1 

Remote  Only  !  !  1 

Physical  Portability 

Movable  among  ATE  Stations  I  !  3 

Transportable  to  Different  Sites  !  !  3 


Electrical  Signals 

Bypassed  (via  bus  !  x  I 

inter  face) 

Nominal  Values  !  !  1 

(not  accurate) 

Accurate  Values  I  I  2 

Accurate  Signals  I  ]  3 

+  noise  effects 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


4.  Time  Requirements 


ReaL  Time  (UUT  Operation) 

Full  Real  Time 
Scaled,  Reduced  time 
Asynchronous 

Block  Time  (How  Quickly 
How  Long) 

Continuous  Usage 
Allocatable  by  Block 
Time 

Access  Requirements 
On  Demand 
Flexible  Schedule 
Fixed  Schedule 

Response  Time  (Validators 
Perspective) 

No  Perceptible  delays 

from  real  UUT  operation 
Up  to  2  Min  delays 
(or  5  sec/step) 

Familiarization  Time 
No  Training  Required 
(Built  in  Aids) 

2  Day  Training  + 

Month  Familiarization 
About  as  long  as 
learning  ATE  Use 


1 

1 


!  1 
I  3 


I 

J 

I 


3 

1 


2  ! 


3 


l 


3 

1 

2 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


5.  V  and  V  Session  Functional  Requirements 


5.1  Initial ization 

System  Initialization 
Display 

Panel  Switches 
Restart  at  Saved  State 

UUT  Model  Identification 
Verify  Proper  Model 
Provide  Parameter 
input  for  model 

ATE  Interface  Model  Identification 
Verify  Proper  Model 
Provide  Parameter 
Input  for  Model 

Operator  Interface 

Initialize  Simulator 
Unique  Functions 
(eg.  Starting  Collection) 
Select  Operation  Mode 


x 

x 

X 


5.2  Interactive  Simulation  Session 
5.2.1  Input  Requirements 

•*»  •  •  •  •  •  f  4  «  •  .  rtf. 

Input  Sensing  Requirements 
UUT  Panel  /  Manual  Inputs 


Soft  Keys 

!  x 

Test  Points 

1 

l 

3  ! 

Dynamic  (Manually 

1 

1 

2  I 

Controlled)  Model  Parameter 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


ATE  Signal  Routing  i  x 

Interface  Signals 

ATE  Inputs  I  x 

*»»  Recognize  Valid  .  i  J  2 

Signal  Gen.  Command 

UUT  Environment  Simulator  \  1  2 

Cooling  Monitor 


5.2.2  Output  Requirements 

5.2.2. 1  Calculate  Outputs  Based  on 

ATE  Inputs  !  x  i 

Operator  Inputs  on  Simulated  1  x  ! 

Panel/Control s 

Sequence  of  Input  States  i  !  1 

Functional  Simulation  Model  1  i  1 

(Digital) 

Functional  Simulation  Model  !  '!  1 

( Analog) 

Logic  Model  (Digital)  i  i  2 

Detailed  Circuit  Model  i  !  2 


5. 2. 2. 2  Calculate  Responses  based  on 
Malfunction  Simulation 

Single  Malfunction  !  x  1 

Multiple  Sequential  1  i  3 

Mai funct ions 

Multiple  Concurrent  i  i  1 

Malfunctions 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


5. 2. 2. 3  Generate  Outputs 

UUT  Simulated  Indicators 

Alphanumeric  Representation  lx!  1 

Graphic  Representation  !  !  1  ! 

Outputs  to  ATE  Interface 

Emit  Realistic  Responses  1  x  I  I 

Check  validity  of  Measurement  !  I  2  1 

Command 

5.2.3  Simulation  Status  Outputs 

5.2.3. 1  Information  Output 

Input  State  from  ATE 
Output  State  to  ATE 
UUT  Indicator  States 
Selected  Computed  Values 
Internal  UUT  States 
History  of  States 


5. 2. 3. 2  Output  Method 

Console  CRT 
Printer 

On  Mass  Storage 


5.2.4  Performance  Monitoring 

Active  Interface  Pins 
Signal  Ranges  on  Interfaces 
Closeness  of  signal  limits  to 
bounds  limits 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


5.2.5  On-line  Performance  Evaluation 

Unused  Interface  Pins 
Average  Closeness  of  Signal  to 
bounds  Values 
Fault  Coverage 


5.2.6  Procedural  Branching  Control 

Manual  Control  of  Response 
to  Force  Test  Branching 
Control  of  Branching  via 
Stored  Sequences 
•  Control  of  Branching  via  UUT 
Parameter  Selection 
Control  of  Branching  via 
Malfunction  Selection 

5.2.7  Patch  and  Retain  Temporary  Changes 


Change  Parameters  !  x  ! 

Change  UUT  Response  !  x  ! 

Procedure 

Change  UUT  Model  !  x  ! 

Incrementally 

Substitute  UUT  Model  1  1  3 

Com ponents 

Store  Model  for  Later  !  x  ! 

Recall 


5.3  Interactive  Session  Control 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


5.3-1  Freeze/Run/Replay 

Freeze  State  from 
Simulator  Console 

Freeze  State  From 
ATE  Command 

Automatically  Trigger  Freeze  on 
UUT  Simulator  Bounds  Error 
Expected  Sigral  Error 
After  Specified  no.  actions 
Programmed  Condition 

Freeze  and  Restart 
After  Fixed  Delay 
On  Operator  Initiation 

Save  Frozen  State  on 
Command  from  Console 

Save  Frozen  State  on 
Command  from  ATE 

Restore  Previous  State 

5.3.2  Monitoring  Level  Control 

Trace  Control 

Based  on  Console  Level 
and  Coded  Threshold 
Based  on  Programmed 
Conditions 

5.3.3  Utility  Functions 

Soft  Keys  for  UUT 
Simulator  Control 


I  l 

I  I 


I 

I 

I 


2  ! 


2  ! 


I 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


5.3-4  UUT  Simulator  System  Messages 

Simulator  Error  Conditions 

System  Status 

System  Error  Conditions 

5.4  Performance  Constraints 

5.4.1  Batch  Performance 

Less  than  1  hour  turnaround 

5.4.2  Response  Time  Constraints 

Less  than  2  seconds  in 
Single  Step  Mode 


1  7  1 

i  i  i 

1  7  1 

i  5  i 


5.5  Simulation  Session  Preparation 

5.5.1  Function  Selection 

Select  Function  from  a  Function 
Menu  .  . 

UUT  to  be  Simulated 
Monitoring  Modes 
Interaction  Modes 

5.5.2  Malfunction  Selection 

Select  Specific  Malfunction 
Procedures 

Select  Parameters  Inducing 
Malfunctions 

5.5.3  Procedure  Selection 

Select  Response  Procedures 
Build  New  Response  Procedures 
and  Chain 


x 

x 

X 
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TABLE  2 


A  SUMMARY  OF  UUT  SIMULATOR 
REQUIREMENTS  RESULTS  (continued) 


o  .  5  .  4  Simulation  Pr  epar  at  ion/ Se  lec  t  ion 

Select  Simulation  Detail 
Level 

**  Select  Specific  Simulation 
Select  Simulation  Parameters 


o.S.O  Trace  Level  Selection 

Basel  on  Preset  Level 

Based  on  Programmed  Conditions 


x 


l  1 

I  I 

I  I 

I  I 


3 

2 


o . t  Post-Simulation  Evaluation 

o.o.l  Test  Program  Evaluation 

I e n e r a t e  Statistics  an 

OoverageCSteps,  Pin  Osage  etc.) 
Tolerance  Limits 
Error  Summaries 

Generate  1  Simulated  UUT 
Statistics 

'.enerate  Simulator  System 
Statistics 

5.6.2  Save  Performance  Reports 

On  Mass- storage 

Changes  from  stored  reports 

5.6.3  Display  Specific  Reports 

On  Display 
On  Printer 

Compared  to  Saved  Reports 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (continued) 


6.  Simulation  System  Control  Support 
6. Initialization  J  x  ! 

6.2  Operational  Assurance  !  x  ! 

6.3  Subsystem  Switching  Control  !  !  1 

6.4  Off-line  Activity  Control  !  |  2 

6. 5  System  Instrumentation  !  1 

6.6  System  Recovery  !  !  1 

6.7  Configuration  Management  Support  !  x  ! 

6.8  Performance  Data  Reduction  Support  !  !  1 

6.9  Load  Projection/Enhancement  Support  !  !  1 

6.10  Data  Base  Support 

UUT  Models  !  x  ! 

ID  Characteristics  ]  ]  3 

ATE  Interface  Configuration  !  !  2 

6.11  Off-Line  Preparation/Maintenance 

6.11.1  Model  and  Procedure  Preparation 
Support 

Interactive  !  x  ! 

English-Language  Based  !  x  ! 
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TABLE  2.  A  SUMMARY  OF  UUT  SIMULATOR 

REQUIREMENTS  RESULTS  (concluded) 


6.11.2  Model  and  Procedure  Code  Modifications 

Incremental  Modifications  !  x  |  ; 

6.11.2  Interface  Model  Modifications  lx!  ; 

6.11.4  UUT  Simulator  Verification 

Sy nt ax  Checks  ;  13! 

Inconsistency  Analysis  1  11  I 
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BASELINE  REQUIREMENTS  DEFINITION 


This  section  contains  a  discussion  of  the  set  of  UUT  simulator  requirements 
which  are  in  the  baseline.  This  set  of  requirements  was  determined  to  be 
needed  in  order  to  realize  the  benefits  expected  of  a  UUT  simulator  for 
use  in  support  of  the  V&V  of  test  programs. 

The  requirements  discussions  follow  the  structure  as  defined  in  Section  3, 
Requirements  Collection  Methods.  Only  those  items  in  Table  2  which 
contain  an  "x"  in  column  1  are  discussed  here. 

UUT  Simulator  Start  Time  (R.  1) 


After  the  UUT  Has  Been  Built--Based  on  comments  from  papers  in  the 
literature  and  from  responses  to  our  questionnaire,  it  can  only  be  assumed 
that  the  UUT  simulator  is  started  after  the  UUT  has  already  been  built.  It 
would  be  desirable  to  start  the  development  of  the  UUT  simulator  during 
the  UUT  development  to  use  as  much  of  the  available  design  information 
as  possible,  particularly  when  the  test  programs  are  developed  by  an 
organization  other  than  the  developers  of  the  UUT;  this  has  not  been  done. 
One  of  the  major  problems  in  test  program  development  is  that  not  enough 
information  is  generally  available  on  the  UUT  to  expeditiously  develop  the 
test  pro'graThs'.  Usually  a  sequence  of  iterations  of  information  collection 
is  needed  before  the  test  program  development  can  be  completed. 

To  maximize  the  utility  of  the  simulator  as  an  independent  check  of  the 
test  programs  during  V&V,  it  is  highly  desirable  that  the  simulator 
simulate  the  UUT  as  closely  as  possible.  The  UUT  simulator  should 
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therefore  be  based  on  ITT  design  information  such  as  that  contained  in  a 
Part  II  specification  and  a  schematic.  For  an  independent  check,  it  is 
particularly  important  that  the  l  I  T  simulator  not  be  based  on  the  require¬ 
ments  in  the  TRD  or  the  code  in  the  ATLAS  test  program  developed  to  test 
the  ITT. 

Development  of  the  UUT  simulator  independently  of  the  test  program 
development  can  serve  as  a  secondary  check  ©n  the  adequacy  of  the  infor¬ 
mation  about  the  T'UT.  The  developers  can  cross-check  their  understanding 
of  the  UUT  when  the  test  programs  are  finally  run  against  the  UUT  simulator. 

Modeling  Levels  (R.  2) 


Interface  Level-- 

Recognize  Stimuli  and  Generate  Response  (From  Bus)--This  level  of 
modeling  would  recognize  that  a  specific  stimuli  is  being  applied  (via  the 
ATE  equipment  command  and  the  routing  which  is  in  effect).  The  recog¬ 
nition  assumes  the  ATE  is  based  on  a  bus  structure  architecture  and  that 
the  UUT  simulator  can  intercept  and  respond  on  the  bus  to  mimic  the 
stimuli  generators  and  instruments. 

Uepending  on  the  ATE  approach,  this  level  of  modeling  appears  to  provide 
most  of  the  expected  benefits  and  is  reasonably  feasible. 
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Additional  External  Models- - 


Interface  Adapter  Model:  Topology- -The  topology  of  the  interface 
adapter  must  be  capable  of  being  simulated  to  support  checking  of  proper 
signal  routing  during  the  simulation. 

Operator  Interface  Model- -Any  interface  between  the  UUT  and  the 
operator  (such  as  switches  and  indicators)  which  the  operator  must  observe 
or  set,  must  also  be  capable  of  being  simulated.  See  the  requirement 
below  on  "soft  keys"  ("Utility  Functions"). 

Modeling  Language- - 

English  Language  Based  (ATLAS-like)--If  any  form  of  UUT  simulator 
modeling  language  is  required  (even  if  only  very  simple),  it  must  be 
English  Language  based  and  similar  in  intent  to  ATLAS.  The  primary 
objective  is  to  make  the  resulting  models  easy  to  read  and  easy  to  alter 
for  an  appropriately  skilled  model  developer. 

Fault  Simulation  Level- - 

Interface  Level- -Faults  must  be  capable  of  being  simulated  in  order 
to  force  the  test  programs  into  their  no-go  and  diagnostic  paths.  However, 
as  a  baseline  requirement  it  is  necessary  that  only  the  faults  be  simulated 
at  the  interface  level  with  interface  characteristics  which  ti'igger  the  fault 
detection  tests  in  the  test  programs. 


Physical  Requirements  (R.  3) 

Hardware  Reliability- - 

Commercial--The  hardware  which  is  used  to  implement  the  TJL'T 
simulator  should  have  about  the  same  reliability  requirements  as  the 
.ATE  system.  High  quality  commercial  equipment  is  adequate. 

Electrical  Signals- - 

Bypassed  (via  bus  interface )— Comments  from  the  people  surveyed 
and  comments  from  papers  in  the  literature  place  little  credibility  in 
achieving  adequate  quality  electrical  signals  without  excessive  cost.  Also, 
there  is  a  feeling  that  the  effort  required  to  verify  that  the  simulator  is 
functioning  properly  at  the  signal  level  may  be  too  costly  in  terms  of  the 
developers'  time. 

Therefore,  the  baseline  should  have,  at  most,  the  signals  represented 
digitally  over  the  buses  connecting  the  ATE  central  computer,  the  ATE 
instruments,  stimuli,  and  the  Interface  Adapter  to  the  UUT  simulator 
If  a  software  solution  to  the  HUT  simulator  is  shown  to  be  feasible,  even 
this  bus  interface  may  not  be  necessary. 


Time  Requirements  (R.  4) 

Real  Time  (UUT  Operation)-- 

Asynchronous- -Neither  real  time  or  even  scaled  real  time  are  base¬ 
line  requirements  for  the  UUT  simulator.  The  combination  of  the  UUT 
simulator  and  the  ATE  system  can  operate  synchronously  at  their  natural 
speed. 

Access  Requirements- - 

Flexible  Schedule--The  simulator  should  be  available  to  the  test 
program  developers  and  validators  on  a  flexible  schedule  basis.  On-demand 
scheduling  is  highly  desirable,  but  not  a  baseline  requirement.  Putting  the 
simulator  on  a  fixed  schedule  is  too  constraining. 

V&V  Session  Functionality  (R,  5) 

Initialization  (R.  5.1)-- 

System  Initialization -- 

•  Display- -All  displays  must  be  capable  of  being  initialized. 

•  Panel  Switches-- All  panel  switches  must  be  capable  of  being 
put  into  an  initial  condition  either  automatically  or  via 
instructions  to  the  operator. 

•  Restart  at  Saved  State--It  must  be  possible  to  restart  a 
simulation  (including  all  displays  and  external  switches)  from 
a  saved  state. 
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UUT  Model  Identification- - 


•  Verify  Proper  Model- -It  must  be  possible  to  verify  that 
the  proper  model  of  the  desired  UUT  is  going  to  be  used. 

•  Provide  Parameter  Input  for  Model- -During  initialization  it 
must  be  possible  to  provide  for  key  model  parameter  inputs, 
which  allows  the  user  to  make  the  next  simulation  run  unique 
from  previous  runs. 

ATE  Interface  Model  Identification- - 

•  Verify  Proper  Model--If  the  ATE  Interface  is  to  be  simulated, 
it  must  also  be  possible  to  verify  that  the  proper  model  is 
going  to  be  used. 

•  Provide  Parameter  Input  for  Model- -P  must  be  possible  to 
provide  for  key  model  parameter  inputs  for  the  ATE  model, 
if  the  ATE  is  to  be  modeled. 

Operator  Interface- - 

•  Initialize  Simulator  Unique  Functions- -It  must  be  possible  to 
initialize  any  unique  function  which  the  UUT  simulator  has 
that  involves  interfacing  with  the  operator.  Presetting 
switchings  and  making  simulated  jumper  connections  are 
examples. 

•  Select  Operation  Mode--A  number  of  alternative  operator 
interface  modes  may  be  available  (c.g.,  totally  manual,  semi¬ 
automatic,  or  totally  automatic).  During  initialization  it 
must  be  possible  to  choose  the  proper  mode  for  the  session. 
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Interactive  Simulation  Session  (R.  5.  2)-- 


Input  Requirements  (R.  5.2 . 1 

•  Input  Sensing  Requirements 

--UUT  Panel/Manual  Inputs- -The  UUT  simulator  must  be 
capable  of  sensing  "soft  keys"  which  mimic  manual  inputs 
and  actions  which  would  be  performed  on  the  real  UUT. 

--ATE  Signal  Routing- -Depending  on  the  simulator,  it  may  be 
necessary  to  simulate  the  Interface  Adapter  and  the  signal 
routing  which  the  ATE  is  forcing. 

--Interface  Signals--The  UUT  simulator  must  be  capable  of 
handling  representations  of  input  signals  from  ATE  stimuli 
generators  appropriate  for  the  recommended  level  of 
simulation. 

Output  Requirements  (R.  5.  2.  2)-- 

•  Calculate  Outputs  Based  on  (R.  5.  2.  2. 1 ) 

--ATE  Inputs- -The  simulator  must  be  capable  of  calculating 
outputs  based  on  input  conditions. 

--Operator  Inputs  on  Simulated  Panels /Controls --It  must  also 
be  possible  to  generate  outputs  based  on  the  state  of 
simulated  panels  and  controls  which  may  have  to  be  man¬ 
ipulated  by  the  operator  because  of  operator  action  requests 
in  the  test  program. 
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Calculate  Responses  Based  on  Single  Malfunction 
Simulation  (R.  5.  2.  2.  2) 


--Single  Malfunction- -It  must  be  possible  to  have  the  CUT 
simulator  mimic  the  occurrence  of  a  single  malfunction 
which  results  in  the  appropriate  symptom  in  the  simulator 
responses. 

•  Generate  Outputs  (R.  5.  2.  2,  3) 

--UUT  Simulated  Indicators 

Alphanumeric  Representation- -If  the  UUT  has  indicators 
which  the  operator  is  required  to  read  or  react  to,  the  CUT 
simulator  must  be  capable  of  outputting  the  states,  at  least 
on  an  alphanumeric  display.  It  is  not  considered  necessary 
to  output  the  information  graphically  or  on  a  realistic 
panel.  It  is  assumed  that  the  test  program,  developer  or 
validator  is  adequately  trained  in  making  the  alphanumeric 
to  indicator  status  translation. 

--Outputs  to  ATE  Interface 

Emit  Realistic  Responses- -It  must  be  possible  for  the 
simulator  to  emit  realistic  responses  appropriate  to  the 
level  of  simulation  fidelity  recommended. 

Procedural  Branching  Control  (R.  5.  2.  6)--It  must  be  possible  to 
control  the  branching  in  test  programs  via  the  following  methods: 

•  Manual  Control  of  Responses--It  must  be  possible  to  allow 
the  user  of  the  UUT  simulator  to  manually  control  the 
simulator  response  in  order  to  force  the  program  to  take 


a  desired  branch.  This  capability  is  analogous  to  making  a 
manual  alteration  in  a  real  UUT  to  force  test  program 
branching. 


•  Control  of  Branching  via  Stored  Sequences- -It  mus’  be  possible 
to  force  the  test  program  being  exercised  to  take  a  sequence 

of  actions  in  the  UUT  simulator.  These  stored  sequences 
would  typically  be  used  to  revalidate  a  sequence  of  tests  which 
had  been  operating  correctly  before  a  modification  to  the 
test  program  was  made.  The  capability  is  required  to  ensure 
that  test  program  modifications  do  not  have  unexpected  and 
unintended  side  effects. 

•  Control  Branching  via  UUT  Parameter  Selection- -It  must 
be  possible  to  allow  parameters  in  the  UUT  simulator  to 
control  branching  in  the  test  program.  These  parameters 
would  be  set  at  initialization  time  or  dynamically  during  a 
simulation. 

•  Control  of  Branching  via  Malfunction  Selection--It  must  be 
possible  to  control  the  branching  in  a  test  program  by 
selecting  specific  simulated  malfunctions  in  the  UUT. 

Patch  and  Retain  Temporary  Changes  (R.  5.  2.  7 ) — It  must  be  possible 
to  make  temporary  patches  to  the  UUT  simulator  in  order  to  make  it 
respond  differently.  The  reason  for  this  may  be  that  a  more  detailed  model 
is  required,  that  the  original  response  was  not  correct,  or  that  a  trial  is 
being  made  to  see  how  the  test  program  will  respond.  After  it  is  verified 


that  temporary  change  is  to  be  made  permanent,  it  must  be  possible  to 
retain  the  change  in  permanent  storage.  The  following  areas  of  change 
must  be  supported: 


•  Change  Parameters  of  the  PUT  Simulator 

•  Change  UUT  Response  Procedure 

•  Change  UUT  Model  Incrementally 

•  Store  Model  for  Later  Recall 

Interactive  Session  Control  (R.  5.  3)-- 

Freeze/  Run/ Replay  (R.  5.  3,  1 )-- 

•  Freeze  State  From  Simulator  Console--It  must  be  possible 
to  freeze  the  state  of  the  UUT  simulator  and  of  the  ATE  from 
the  simulator  console.  The  purpose  of  this  freeze  is  to  allow 
the  user  to  examine  the  state  of  the  UUT  simulator  and, 

if  desired,  to  allow  him  to  change  or  restart  the  simulation. 

•  Freeze  and  Restart 

--On  Operator  Initiation- -Control  of  restart  must  be 
exercisable  from  the  UUT  simulator  control  console. 

•  Save  Frozen  State  on  Command  from  Console--A  command 
must  be  supported  to  save  the  state  of  the  UUT  simulator 
after  a  freeze.  The  purpose  of  saving  the  state  is  to  allow 
a  later  restart  at  the  same  state.  This  feature  is  required 
to  minimize  the  time  required  to  get  back  to  a  point  of 
interest  in  the  test  program  debugging  or  verification. 
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•  Select  Function  from  a  Function  Menu--During  simulation 
preparation  it  must  be  possible  to  select  functions  from  a 
function  menu.  The  following  are  specific  areas  of 
functionalities  which  must  be  selectable: 

--UUT  To  Be  Simulated--It  is  assumed  more  than  one  UUT 
may  be  supported  by  one  simulator  system  and  that  multiple 
versions  of  a  specific  UUT  simulator  may  be  present. 
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--Monitoring  Modes- -Determining  the  amount  of  tracing  and 
checking  (if  implemented). 

--Interaction  Modes- -Determ nig  how  the  simulation  is  to  be 
run  (from  completely  interative  to  batch  oriented). 

Malfunction  Selection  (R.  5.  5.  2)-- 

•  Select  Specific  Malfunction  Procedures  and  Select  Parameters 
Inducing  Malfunctions--  During  a  simulation  session  preparation, 
it  must  be  possible  to  select  speciiic  malfunction  procedures 
which  will  force  test  program  branching.  It  must  also  be 
possible  to  select  UUT  parameters  which  will  force  mal¬ 
function  symptoms  during  the  simulation  session. 

Procedure  Selection  (R.  5.  5.  3)-- 

•  Select  Response  Procedures-  -it  :  •  r-sumed  that  a  compre¬ 
hensive  set  of  procedures  win  o.  a: c  umulated  to  simulate 
the  UUTs  responses  to  test  program  actions.  During  a 
session  preparation,  it  must  be  possible  to  select  sets  of 
these  response  procedures  to  specify  how  the  UUT  simulator 
will  respond  during  the  simulation  session. 

•  Build  New  Response  Procedures  and  Chain-- It  must  also  be 
possible  to  build  new  response  procedures  for  the  UUT 
simulation.  Incrementally  building  up  such  response 
procedures  must  be  one  way  of  grad-,  ally  building  a  simulator 
for  a  specific  UUT. 
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Simulation  Preparation/Selection  (R.  5.  5.4)-- 


•  Select  Simulation  Detail  Level,  Select  Specific  Simulation, 
and  Select  Simulation  Parameters--If  various  levels  of  detail 
are  possible  in  a  UUT  simulation,  it  must  be  possible  to 
select  the  desired  sets  to  be  used  in  the  session.  Specific 
simulation  models  used  previously  must  also  be  selectable. 
Parameters  which  control  the  simulation  must  also  be 
selectable. 

Simulation  System  Control  Support  (R.  6) 

Initialization  (R.  6,  1  )--The  UUT  simulator  system  must  be  capable  of  being 
initialized  in  a  simple  manner  after  power  is  turned  on.  The  normal 
computer  operating  system  procedures  for  initialization  are  assumed  to 
be  adequate. 

Operational  Assurance  (R.  6.  2)--The  simulation  system  must  have  resident 
in  its  file  system  a  set  of  programs  which,  when  executed,  will  ensure 
that  the  system  is  ready  and  performing  adequately  to  run  the  simulator. 

Configuration  Management  Support  (R,  6.  7)--The  simulator  system  must 
contain  a  file  system  and  operating  system  which  are  adequate  for  config¬ 
uration  management  support  of  all  of  the  software  components  which 
make  up  the  UUT  simulator  system  and  specific  UUT  simulators. 
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Data  Base  Support  (Ft.  5.  10)-  - 

UUT  Models- -The  simulator  system  must  be  capable  of  supporting 
the  UUT  model  data  base  at  minimum. 


ENHANCEMENTS 


This  section  contains  descriptions  of  desirable  UUT  simulator  enhancements 
beyond  those  required  in  the  baseline.  The  list  of  enhancements  is  included 
in  Table  2  as  explained  in  Section  3,  Requirements  Collection  Methods. 

The  descriptions  below  provide  more  details  about  the  enhancements  and 
the  rationale  for  the  weighting  which  was  assigned  to  them.  Only  those 
items  in  Table  2  which  are  classified  as  enhancements  are  described  below. 

UUT  Simulator  Start  Time  (R.  1  ) 


At  UUT  Design--The  id.  a!  lime  'o  -tart  the  definition  of  a  UUT  simulator 
would  be  late  in  the  desur  of  die  l  UT  when  most  of  the  design  decisions 
have  been  frozen.  It  is  at  this  time  that  the  design  decisions  are  still 
fresh  in  the  minds  of  the  developers  and  much  of  this  information  could 
be  put  into  an  appropriate  form  lor  the  UUT  simulator.  If  circuit  models 
were  used  in  the  development  of  the  UUT,  the  information  from  these 
models  could  be  transform  ed  mto  information  needed  in  a  UUT  simulator 
to  support  test  program  development. 

Note  that  the  information  m  the  models  of  the  UUT  needed  for  design 
verification  is  different  trum  the  information  needed  for  test  program 
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development.  But  it  can  be  conjectured  that  the  information  needed  for 
test  program  development  could  be  derived  from  that  in  the  design 
verification  model. 

.A  value  of  2  was  assigned  to  this  feature  to  reflect  the  anticipated 
difficulty  and  credibility  in  being  able  to  start  on  the  UUT  simulator  so 
early  in  the  design  of  a  UUT.  Also,  if  too  much  detail  is  used,  it  may  get 
in  the  way  of  supporting  test  program  debugging  and  verification.  Keeping 
up  with  UUT  design  changes  is  also  a  potential  problem. 

Modeling  Levels  (R.  2) 

Circuit  Levels-- 


Analog-  - 

•  Topology  and  Simple  Models--Modeling  at  this  level,  if 
possible,  could  still  be  relatively  fast.  But  reservations 
about  the  adequacy  and  payoff  versus  the  anticipated  effort 
were  expressed.  A  value  of  2  was  assigned  because  of 
these  reservations. 

•  Topology  and  Good  Models- - Though  the  modeling  would  be 
better  than  above,  the  resulting  speed  is  expected  to  be  lower. 
Furthermore,  the  same  reservations  about  adequacy  and 
payoff  still  hold.  A  value  of  2  was  assigned. 
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•  Good  Models  and  Second  Order  Coupling  Effects- -Even  with 
good  models,  second-order  coupling  effects  would  still  be 
needed  to  model  some  types  of  analog  failures.  The  added 
complexity  and  reduced  simulation  speed  are  not  considered 
effective  for  a  CUT  simulator  to  support  test  program  develop¬ 
ment.  A  value  lower  than  1  should  have  been  assigned,  but 
because  of  the  convention,  a  1  had  to  be  used. 


Digital- - 

•  Register  Transfer  Model-- A  CUT  simulator  embodying  a 
register  transfer  model  of  the  internal  CUT  functionality  is 
considered  more  v  seful  than  detailed  models  for  the  analog 
case.  Such  a  model  should  be  particularly  useful  for  LTTs 
which  are  too  complex  for  automatic  test  program  generation 
application. 

The  utility  of  a  register  transfer  model  to  support  test 
program  debugging  and  verification  is  still  questionable.  It 
is  possible  to  perform  such  modeling  independent  of  the 
ATE  system  so  that  when  the  test  vectors  or  stimuli  patterns 
are  applied,  the  results  need  not  Ik  questioned  as  much  (as 
is  the  case  if  the  test  vectors  are  g<  nerated  by  ATPG).  For 
these  reasons  a  weight  of  2  was  assigned. 

•  Register  Transfer  Model  and  Timing  Model- -This  is  a 
higher  fidelity  version  of  the  case  discussed  above.  The 
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resulting  model  would  be  slower  but  more  useful.  But  the 
same  relative  utility  question  can  be  raised,  and  consequently 
a  weight  of  2  was  assigned. 

•  Detailed  Circuit  Model--Models  such  as  those  used  for 
automatic  test  program  generation  are  too  detailed  for  test 
program  development  general  usage.  A  value  of  l  was 
assigned. 

Interface  Level--  . 

- — .  *  ■ — --  •  .  . 

Recognize  Stimuli  and  Generate  Accurate  Electrical  Signals- -For  this 
case  the  recognition  would  depend  on  recognizing  signals  across  UUT  pins 
and  generating  actual  electrical  responses.  To  achieve  adequate  fidelity, 
the  user  would  have  to  be  concerned  about  many  signal  details  which  are 
judged  as  getting  in  the  way  of  doing  test  program  debugging  and  verification. 
A  value  of  1  was  assigned. 

.Additional  External  Models- - 

Interface  Adapter  Models- - 

•  Circuits- -The  additional  complexity  of  this  level  of  I A 
modeling  is  judged  to  get  in  the  way  of  doing  test  program 
debug  and  verification.  A  value  of  1  was  assigned. 


55 


ATE  Equipment  Model- -An  equipment  model  is  particularly  useful 
if  a  software  UUT  simulator  solution  is  to  be  used  or  if  the  ATE  system 
is  based  on  a  bus  architecture  and  the  LET  simulator  is  interfaced  to  the 
ATE  on  this  bus.  A  value  of  3  was  assigned. 

Modeling  Language- - 

Graphical- -A  graphical  communication  of  the  UUT  internal  model 
and  interface  model  would  be  useful  in  some  cases  to  minimize  develop- 
'ment  time.  But  because  of  the  extra  effort' needed  to  use  it  and  because 
it  would  not  be  as  universally  applicable  as  an  English  Language  based 
system,  a  value  of  2  was  assigned. 

Fault  Simulation  Capability- - 

Functional  Model  Level- -Introducing  simulated  faults  modeled  by 
functional  modules  appears  desirable.  The  value  above  what  can  be  done 
beyond  that  at  the  Interface  Level  for  supporting  test  program  debugging 
and  verification  is  an  issue.  A  value  of  2  was  assigned. 

Circuit  Level- -Supporting  faults  simulation  for  the  full  UUT  at  the 
circuit  level  appears  too  detailed  to  be  effective  in  debugging  and  verifying 
test  programs,  A  value  of  1  was  assigned.  Circuit  level  simulation 
may  be  useful  in  a  multi-level  simulator. 
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Physical  Requirements  (R.  3) 


Hardware  Reliability-- 

Militarized- -Using  militarized  components  and  standards  is  not 
warranted.  A  value  of  1  was  assigned. 

Ruggedized-- Even  ruggedized  requirements  are  too  stringent  to  be 
requirements  for  the  UUT  simulator  implementation.  /V*value  of  2  was 
assigned. 

Location- - 

Adjacent  ATE- -It  is  most  desirable  that  the  UUT  simulator  be  capable 
of  being  placed  adjacent  to  the  ATE  system  with  which  it  is  interacting. 

The  final  form  of  the  UUT  simulator  will  determine  how  valuable  the 
relative  location  is.  A  value  of  3  was  assigned. 

Remote  and  Real-Time  Access--If  access  to  the  simulator  must  be 
close  to  real  time,  having  the  UUT  simulator  implemented  on  a  remote 
computer  is  not  desirable.  A  value  of  1  was  assigned. 

Remote  Only- -If  the  simulator  is  strictly  software,  it  could  be 
implemented  on  a  remote  computer.  Feedback  from  our  survey  indicated 
that  this  is  not  very  desirable.  A  value  of  1  was  assigned. 
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Physical  Portability- - 

Movable  Among  ATE  Stations- -It  is  desirable  that  the  simulator  be 
capable  of  being  rolled  to  allow  positioning  beside  different  ATE  stations. 
A  value  of  3  was  assigned. 

Transportable  to  Different  Sites- -Ease  of  transporting  the  simulator 
or  making  it  easily  accessible  to  different  sites  facilitates  support  of  TPS 
development  by  organizations  different  from  the  developers  of  the  PUT. 

A  value  of  3  was  assigned. 

Electrical  Signals- -If  electrical  signals  are  to  be  generated  (particularly 
in  the  analog  case),  they  must  be  accurate  and  realistic  to  be  useful  to 
support  debug  and  verification  of  test  programs.  Therefore,  a  simulator 
which  generates  only  nominal  values  was  assigned  a  value  of  1;  one  that 
produces  accurate  signals,  a  value  of  2;  and  one  that  produces  accurate 
signals  and  noise  effects,  a  value  of  3.  Cost-effectiveness  analysis 
is  expected  to  show  that  the  latter,  particularly,  is  not  feasible. 

Time  Requirements  (R.  4) 

Real  Time  (UUT  Ope  ration)- -Full  real  time  or  even  scaled  time  are  not 
considered  high  needs  for  test  program  debug  and  verification.  A  value 
of  1  was  assigned  to  both. 

Block  Time-- Allocation  of  the  UUT  simulator  by  block  time  appears  ade¬ 
quate  and  was  assigned  a  value  of  3.  Having  the  simulator  available 
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on  a  continuous  basis  (for  example,  by  dedicating  a  system  or  by  time- 
shared  support)  would  be  nice,  but  it  was  not  judged  a  strong  need.  A 
value  of  1  was  assigned. 

Access  Requirements--Having  the  simulator  available  on  demand  would 
be  useful,  particularly  during  test  program  debug,  but  it  is  not  a  require¬ 
ment.  A  value  of  3  was  assigned.  A  fixed  schedule  is  less  desirable 
because  it  may  cause  developers'  time  to  be  wasted.  Therefore,  a  value 
of  1  was  assigned. 

Response  Time- -Having  no  delay  as  compared  to  the  real  ATE  and  real 
UUT  would,  of  course,  be  the  ideal  case.  But  developers  are  conditioned 
to  expect  delays  and  to  plan  for  them.  Therefore,  requiring  no  delay  was 
assigned  only  a  value  of  2. 

To  prevent  wasting  developers'  time,  some  limit,  such  as  two  minutes, 
should  be  a  maximum  wait  goal  for  normal  types  of  operations.  If  the 
developer  is  interacting  directly  with  the  UUT  simulator  (such  as  in 
stepping  through  a  program)  no  more  than  a  five  second  delay  should  have 
to  be  tolerated. 

Familiarization  Time- -The  following  values  were  assigned: 

No  Training  Required  (Built  in  Aids):  3 
2  Day  Training  +  Month  Familiarization:  1 
About  as  long  as  learning  ATE  Use:  2 
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V&V  Session  Functionality  (R.  5) 


Interactive  Simulation  Session  (R.  5.2)--  See  Table  2. 

Interactive  Session  Control  (R,  5.  3)--See  Table  2. 

Performance  Constraints  (R.  5.4)- - 

Batch  Performance  (R.  5.  4.  1  )--If  the  simulator  is  to  be  used  in  batch 
mode",  •  it -is  desirable  to  provide  adequate. turnaround  to  allow  a  developer 
to  be  concurrently  analyzing  two  or  three  segments  of  his  test  program. 

In  this  way  he  can  analyze  one  set  of  results  while  waiting  for  the  others 
to  get  back.  A  turnaround  time  of  one  hour  appears  a  reasonable  goal. 

Response  Time  Constraints  (R,  5.  4.  2 )- -If  the  developer  is  interacting 
with  the  FFT  simulator  system  (such  as  editing  or  monitoring  a  trace 
output),  a  response  goal  of  two  seconds  should  be  targeted.  If  the  system 
becomes  much  slower,  the  developer  is  wasting  time  and  may  become 
frustrated. 

Post-Simulation  Evaluation  (R.  5,  6)--See  Table  2. 

Simulation  System  Control  Support  (R,  6) 

Subsystem  Switching  Control  (R.  6,  3)--S.ystems  that  require  high  uptimes 
may  have  to  be  supported  by  dynamic  configuration  control  so  that  a 
faulty  component  can  be  switched  out  and  a  replacement  can  be  switched  in. 


Availability  requirements  for  the  UUT  simulator  are  not  high  enough  to 
warrant  such  complexities.  A  value  of  1  was  assigned  to  this  feature. 


Off-Line  Activity  Control  (R.  6.4)--It  is  sometimes  desirable  to  perform 
background  support  activities  at  the  same  time  that  a  simulation  is  in 
progress.  Though  this  is  desirable  for  the  UUT  simulator,  it  is  not 
essential.  A  value  of  2  was  assigned. 

System  Instrumentation  (Ft.  6.  5)- -To  help  ensure  that  the  system  is  perform¬ 
ing  optimally,  it  is  desirable  to  have  a  good  complement  of  system 
instrumentation.  These  features  are  also  useful  if  it  is  determined  that  the 
performance  of  the  simulator  is  not  good  enough  and  selective  components 
must  be  improved.  The  instrumentation  then  supports  finding  where  the 
performance  problems  are  and  what  improvements  can  be  made. 

System  Recovery  (R.  6.  6)--It  is  desirable  to  have  automatic  recovery 
after  a  power  failure  or  a  system  fault.  Because  of  the  save  and  restore 
features  already  in  the  baseline,  additional  system  recovery  features  are 
less  useful. 

Performance  Data  Reduction  Support  (R.  6,  8)- -It  is  desirable  to  have 
system  support  which  will  help  to  reduce  the  system  performance  data 
and  present  it  in  summary  form. 

Load  Projection/Enhancement  Support  (R.  6.  9)- -If  the  UUT  simulator 
becomes  heavily  used  and  if  multiple  users  are  supported,  features  which 
help  projection  of  loading  on  the  simulator  and  which  help  to  evaluate 


potential  enhancements  (such  as  added  memory  or  added  disc  space)  will 
be  useful  in  getting  maximum  benefit  out  of  the  UUT  simulator. 

Off-Line  Preparation/ Maintenance  (R.  6,  11) — 

UUT  Simulator  Verification  (R.  6.  11.4)—  One  of  the  problems  which 
must  be  addressed  is  verifying  that  the  UUT  simulator  performs  as 
required.  The  generic  support  modules  which  are  used  for  all  implemented 
simulators  must  be  carefully  verified,  as  any  other  software  is.  However, 
the  specific  UUT  simulator  models  also  require  verification  support.  A 
first-cut  capability  would  be  extensive  syntax  checks  of  the  descriptions 
of  the  simulators.  A  more  sophisticated  capability  would  support  tools 
to  anyalyze  potential  inconsistencies. 

REQUIREMENTS  INFORMATION  FROM  LITERATURE 

A  reasonably  extensive  search  was  performed  in  the  public  literature 
which  may  have  a  bearing  on  the  problem  we  are  addressing.  The  references 
listed  at  the  end  of  this  report  are  a  result  of  this  search  and  secondary 
leads  from  papers  which  were  already  found.  We  had  found  additional 
references,  but  because  the  papers  were  not  conveniently  accessible  and 
in  most  cases  of  marginal  relevance,  they  are  not  listed. 

The  papers  were  reviewed  and  classified  according  to  relevance  on  different 
topics.  The  results  of  this  classification  are  presented  in  Appendix  A. 

The  topics  which  were  used  for  classification  were  as  follows: 
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UUT  Reference 


•  ID  (IA)  Reference 

•  TRD  Reference 

•  Analog  UUT 

•  Digital  UUT 

•  Test  Program  Development  Costs  Information 

•  TPS  Development  Difficult  Information 

•'  '  UUT  'Difficulties . . 

•  ID  (IA)  Difficulties 

•  ATE  Difficulties 

•  Payoff  Information 

•  Development  Tools  Descriptions 

Note  that  not  all  references  were  classified.  Some  references  were 
obtained  after  the  classification  was  performed. 

During  Phase  I  of  the  program  these  references  were  viewed  primarily 
in  context  of  what  they  say  about  the  problem  which  we  are  addressing. 
Little  attention  was  paid  to  information  on  potential  alternative  solutions 
or  techniques  useful  for  a  UUT  simulator.  The  papers  will  be  reviewed 
again  in  Phase  II  to  extract  this  type  of  information. 
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The  insights  in  some  of  the  papers  influenced  our  thinking  in  the  development 
of  the  UUT  simulator  baseline  requirements  and  in  the  prioritization  of 
the  enhancements. 


QUESTIONNAIRE  RESULTS 

The  questionnaire  prepared  for  the  survey  attempted  to  answer  many 
questions  without  introducing  the  author's  bias.  This  approach  resulted 
in  a  number  of  questions  that  were  open  to  interpretation  but  unburdened 
'by  specific  solution  biases.  One  *0t  the  goals  df  the  questionnaire  was 
to  get  a  measure  of  the  effect  the  UUT  simulator  might  have  on  various 
phases  of  V&V.  Many  of  the  questions  attempted  to  identify  areas  of  the 
simulator's  functionality  on  which  to  focus  increased  emphasis.  The 
other  major  goal  of  the  questionnaire  was  to  provide  a  forum  for  the 
expression  of  opinion  and  suggestions  for  improvement  to  the  V&V 
process. 

The  questionnaire  approach  was  taken  both  to  get  the  broadest  possible 
coverage  and  to  provide  those  surveyed  with  the  most  time  to  consider 
the  questions. 

I’he  questionnaire  and  summary  responses  to  the  questions  follow. 

\s  a  Test  Program  Validator 


i.  Did  you  have  your  test  program  or  Interconnect  Device  (ID) 
development  delayed  by  the  unavailability  of  a  UUT° 
yes  no 
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If  yes,  what  percentage  of  the  programs  that  you  worked  on  had 
this  problem? 

6  yes  Responses  ranged  from  "a  small  number"  to  "100°^ 

1  no  of  the  test  programming  or  ID  development  efforts  were 
delayed  by  the  unavailability  of  UUTs.  " 

2.  Did  you  have  a  "Golden"  UUT  exhibit  a  failure? 

yes _  no _ 

6  yes  Probably  fewer  than  1% 

1  no 

3.  Did  you  lack  sufficient  time  on  the  ATE  to  debug  your  test 
program  and  ID? 

yes _  no _ 

If  yes,  how  would  you  have  relieved  this  problem? 

5  yes  Provide  "more  stations"  interleave  ID  and  test  program 

2  no  design,  use  a  "simulator",  "do  more  work  off  station.  " 

4.  Did  you  find  a  significant  percentage  of  your  debug  time  devoted 
to  debugging  your  ID? 

yes _  no _ 

If  yes,  what  percentage?  _ 

3  yes  Percentage  of  debug  time  ranged  from  10-50%. 

4  no 

5.  Did  you  have  trouble  getting  what  you  felt  was  the  necessary 
source  data  to  develop  your  test  program  ? 

yes _  no _ 
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If  yes,  describe  the  problems  and  what  you  could  have  done  to 
alleviate  them. 

5  yes  I  Solution:  "Do  the  best  you  can  with  what  you  have.  " 


6.  Did  you  find  that  the  Test  Requirements  Document  (TRD)  did  not 
adequately  test  the  device? 


Was  the  testing  described  incomplete? 

5  yes  Noncommittal  responses  generally  left  judgment  of  the 
adequacy  of  the  TRD  to  the  end  user. 

7.  Did  you  spend  more  time  analyzing  the  UUT  than  would  have  been 
necessary  if  you  would  have  had  a  simulator? 

yes _  no _ 

2  yes  The  remaining  three  responses  expressed  concern  that 
2  no  they  might  be  debugging  the  simulator. 

8.  Did  you  have  trouble  preserving  the  integrity  of  low  level 
signals  ? 


If  yes,  describe  the  problem. 

4  yes  "Station,  UUT,  and  ID  Noise"  were  identified  as  sources 
3  no  of  noise  problems. 


10.  Did  you  have  grounding  problems? 

yes _  no _ 

If  yes,  describe  them. 

4  yes  Grounding  problems  were  attributed  to  an  ID  system  ground 

3  no  inadequacy. 

11.  Did  you  have  trouble  understanding  the  TRD? 

yes  _  no _ 

If  yes,  why ?  ' 

....  *---••  *  *"  '  * ' ' 

4  yes  Different  writing  styles  and  vague  requirements  were  the 
3  no  main  problems  in  understanding  a  TRD. 


What  Were  the  Problems  and  How  Would 
You  Rank  Them  in  Severity? 


A.  Test  Program  Development  Problems 

1.  What  type  of  UUTs  (analog,  digital,  etc.  )  required  the  most  time 
to  debug  and/or  validate?  How  much  time  was  required  for  each? 

A  four  to  one  ratio  of  analog  to  digital  test  program  develop¬ 
ment  time  was  given,  qualified  by  other  responses  indicating 
that  it  depended  on  the  number  of  tests  in  the  TRD. 

2.  What  do  you  believe  the  reasons  are  for  one  type  of  device 
requiring  more  debug/development/validation  time  than  another? 

ATG  was  cited  in  two  responses  as  the  reason  why  digital 
was  easier.  Other  causes  were  length  of  TRD  and 
complexity  of  the  test. 


3.  How  do  you  work  around  these  problen 


Responses  were  generally  "to  dig  into"  the  problem.  .A 
more  thoughtful  response  was  to  create  special  software 
to  permit  more  direct  access  to  test  program  statements 
needing  debug. 

4.  What  tools  or  aids  would  you  like  to  have  had  to  help  solve 

these  problems? 

....  .  .Suggested. tools  yic’u.dftd  a  .system,  .trace  -that  would  keep 

track  of  the  station's  status  (what  stimuli  had  been  applied, 
relays  closed,  etc.  )  as  the  test  progressed,  aids  to  explain 
ATLAS  fields,  analog  ATG,  ATE  simulator,  bus  and  logic 
analyzers,  user  programmable  "soft"  keys,  a  ITT 
simulator,  and  an  automated  ID  checker. 

5.  What  percentage  of  your  test  program  debug  time  was  spent 

on  the  following: 

a.  checking  the  logical  flow  of  the  software? 

b.  verifying  signal  levels,  tolerances,  and  waveshapes  ? 

c.  verifying  the  adequacy  of  the  test  requirements  document? 

d.  verifying  the  operation  of  the  LET  as  assumed  by  your  test 
program  ? 

e.  resolving  grounding  and  noise  problems?  Please  describe 
these  problems  and  aids  that  you  believe  could  relieve  these 
problems. 

f.  resolving  problems  with  low-level  signals? 
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The  following  six  responses  were  received  for  the  distribution  of  debug 
time: 


a. 

checking  S/W  logical  flow 

30% 

1% 

50% 

10% 

60% 

5% 

b. 

verifying  signals 

70% 

75% 

10% 

30%; 

10% 

50% 

c. 

verifying  TRD 

-- 

1% 

5% 

20% 

15% 

10% 

d. 

verifying  TUT  operation 

-- 

13% 

20% 

20%, 

10% 

30% 

e.  . 

f.  Resolving  grounding  and 

-- 

10% 

15% 

20%, 

5% 

5% 

noise  problems 


ID  .Associated  Problems 

1.  What  type  of  problems  did  you  experience  in  checking  out  your 
IDs?  How  much  time  was  spent  on  each? 

Misinterpretation  of  TRD  requirements  and  production 
errors  were  most  commonly  cited  ID  problems. 

2.  Which  of  these  were  the  most  time  consuming? 

Misinterpretation  of  TRD  requirements  was  cited  as  the 
most  time-consuming  problem. 

3.  What  tools  or  aids  would  have  relieved  these  problems? 

An  automatic  ID  checker"  was  the  only  tool  cited.  Other 
suggested  improvements  were  procedural  in  nature. 
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C.  Station  Associated  Problems 

1.  How  significant  was  the  availability  of  time  on  the  ATE  station? 

All  the  responses  indicated  that  availability  of  station  time 
was  very  significant. 

2.  How  much  station  time  was  spent  debugging  IDs? 

Station  time  spent  on  debugging  IDs  was  "very  little,  " 

"15%  of  validation,  "  "50%  too  much"  and  "70%  of  the 
validation  effort.  " 

3.  How  would  it  have  been  useful  to  be  able  to  run  a  test  program 
on  a  software  model  of  the  ATE  station0 

"Very  important;  ",  "only  to  verify.  .  .correct  branching,  " 
and  an  extensive  affirmative  answer  based  on  the  use  of 
a  null  interface  simulation  that  Honeywell  had  provided 
for  his  use. 

4.  How  significant  would  the  operator  interface  presentation  have 
been  for  a  model  of  the  ATE  station? 

fhc  operator  interface  presentation  "must  be  identical,  " 
is  very  significant.  " 

D.  IT  1  Associated  Problems 

l.  How  often  and  how  significant  a  problem  was  the  lack  of  a  ITT 
or  the  failure  of  a  "Golden"  PUT? 

1  ess  than  i  0"'  of  the  total  time"  seems  to  be  the  consensus; 
one  response  said  late  PUTs  were  a  "significant"  problem. 
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2.  What  percentage  of  the  lost  time  in  test  program  development  was 
due  to  incomplete  source  data? 

"Very  little"  to  "50%;"  consensus  seems  to  be  around  10%. 

E.  TRD  Associated  Problems 

1.  What  percentage  of  lost  time  was  due  to  inadequate  test  require¬ 
ments  definition? 

Responses  ranged  from  10%,  to  50%t,- -clustered  the  50%  end. 

2.  Are  there  any  tools  or  aids  that  would  have  helped  relieve  these 
problems  ? 

More  attention  devoted  to  TRDs  including  one  suggestion 

that  they  be  written  in  ATLAS. 

How  Would  You  Validate  a  TPS  if  its  UL'T  Was  Difficult 
to  Obtain  or  its  Availability  Was  Limited? 


A.  What  could  be  done  to  improve  current  validation  efforts? 

1.  What  engineering  data  is  required  to  validate  (and  prove  the 
validation  of)  a  TPS? 

Validate  the  TRD  against  the  program. 

2.  What  are  the  key  parameters  to  be  validated  in  a  TPS? 

Software  sequencing,  signal  transfer  by  the  test  program 
through  the  station  and  ID  to  the  ULT,  TRD  and  test  program 
tests  match. 
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3c  Which  of  these  parameter  s  could  be  validated  without  a  test 
station? 

The  interfaces  (UTJT,  ID,  station)  and  software  logic  flow 
could  be  validated  without  a  test  station. 

4.  How  valuable  would  a  program  which  would  develop  an  interface 
cross-reference  list  from  a  finished  program  be? 

"Great". 

5.  Would  a  program  which  scanned  your  test  program  (or  TRD)  to 
match  its  resources  requirements  to  a  given  ATE  system  be  of 
value  to  you? 

yes _  no _ 

If  not,  why  not? 

Unanimously  yes  with  die  one  cautionary  response  that  the 
resource  allocator  not  detract  from  available  station  time. 

B.  If  a  UIIT  simulator  were  available,  what  attributes  do  you  think  it 
should  have  ? 

1.  What  human  factors  should  be  considered  ? 

a.  Access  via  a  general  purpose  terminal  ? 

yes _  no _ 

b.  A  specialized  modeling/ simulation  keyboard? 

yes _  no _ 
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c.  Librarian  facilities  (storage  and  retrieval  of  models  and 
submodels  for  repeated  use  or  incorporation  into  other 
models)  ? 

yes _  no _ 

d.  .Ability  to  incorporate  user-defined  algorithms  in  the 
simulation  process? 

yes  no  _ 

e.  Other  considerations  ? 

.All  of  the  sub-questions  were  ignored.  Written  comments 
expressed  the  concern  that  the  simulator  might  require 
as  much  validation  as  the  UUT.  .A  more  positive  comment 
was  that  the  simulator  should  not  require  all  of  the 
station  hardware  and  should  produce  a  high-speed 
printout  of  all  station  actions. 

2,  How  much  training  would  you  consider  reasonable  to  learn  to 
use  the  simulator?  How  should  it  be  taught? 

The  same  or  less  than  1/2  the  time  now  required  for  HUT 
test  design. 

3.  At  what  level  of  detail  (complexity)  should  the  simulator  model 
the  UUT? 

The  single  definitive  response  indicated  that  a  simulator 
that  addressed  the  I/O  pins  with  internal  functional  models 
of  circuit  components  would  be  the  appropriate  level  of  detail. 

A  macro-level  model  was  thought  to  be  of  minimum  value. 


73 


.  What  level  of  interface  fidelity  should  the  simulator  have? 

a.  Complete  electrical  and  mechanical  fidelity? 

b.  Ability  to  support  check  out  of  the  interconnect  device0 

c.  An  interface  which  would  simulate  the  loading,  noise,  etc. 
of  the  ATE  on  which  the  test  program  was  to  run? 

d.  An  interface  in  which  pre-defined  (possibly  by  the  user) 
responses  are  supplied  in  response  to  test  program 
measurement  statements? 

Of  two  definitive  responses,  they  agreed  that  complete  electrical 

fidelity  was  required,  and  to  questions  b,  c,  and  d  they  responded 

"more  work,  not  less.  " 

e.  A  simulation  in  which  the  simulator  analyzed  the  test 
program  and  exercised  its  various  diagnostic  paths? 

"Definitely,  with  trace  capability"  and  a  printout  of 
station  actions  on  a  high-speed  printer. 

f.  Can  you  identify  a  preferred  interface  via  an  analogy  with 
an  existing  simulation  system  with  which  you  are  familiar? 

(If  the  system  identified  is  not  widely  available  and  in  the 
public  domain,  we  would  appreciate  any  conveniently 
available  documentation.  ) 

The  "null  interface"  system  provided  to  the  F-15 
site  was  cited  as  a  successful  V&V  aid. 
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5.  Do  you  consider  it  necessary  to  be  able  to  control  the  level  of 
detail  at  which  the  simulation  operates? 

yes  no  _ 

If  not,  skip  to  question  number  6. 

"The  amount  of  data  produced  during  a  given  test  run  should 
be  optional.  " 

6.  How  accessible  must  the  simulator  be? 

a.  Constantly  available? 

b.  Access  scheduled  in  blocks  of  time? 

c.  Should  it  be  interactive? 

One  response  indicated  it  should  be  constantly  available; 
the  other  said  four  hours /day;  both  agreed  it  should  be 
interactive. 

7.  Describe  the  manner  in  which  you  would  see  a  UUT  simulator 
used. 

"This  device  should  be  a  station  simulator,  not  a  UUT 
simulator.  All  of  the  adapter  and  UUT  characteristics 
should  be  provided  by  the  respective  designers  to  be  used 
by  the  simulator.  " 

8.  What  can  be  done  to  ensure  that  the  UUT  simulator  is  cost- 
effective  in  supporting  test  program  verification  and  validation? 

One  response  indicated  that  a  UUT  simulator  would  never  be 
cost  effective;  the  second  suggested  that  the  simulator  be 
"a  combination  program  development  and  station  simulator." 
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TRD  AND  ATLAS  PROGRAM  ANALYSES 


To  present  a  more  in-depth  feel  of  test  program  development  and  veri¬ 
fication,  a  small  representative  set  of  UUT  test  programs  was  analyzed. 
This  section  gives  a  brief  description  of  the  analysis  and  the  results. 

The  test  programs  analyzed  are  items  from  the  F-15  TITE  program.  These 
program  sets  were  developed  by  Honeywell,  based  on  formal  TRDs  from  the 
customer. 


TRD  Analysis 


The  particular  UUT  TRDs  selected  from  TITE  were  the  following: 

•  LVPS  Low  Voltage  Power  Supply 

•  RF  AMP  RF  Amplified 

•  FL  Fault  Locator 

•  LRU-3  Countermeasures  Receiver 


The  following  is  a  brief  description  of  these  units. 

•  LVPS 

The  LVPS  is  a  module  contained  in  the  band  1  and  band  2 
* 

oscillators.  The  low  voltage  power  supply  converts  115  VAC 
400  Hz  3  phase  aircraft  power  to  the  DC  voltages  required  in 
the  band  1  and  band  2  oscillators. 


* 


Band  1  and  band  2  oscillators  are  externally  cooled  LRUs. 


76 


•  RF  .AMP 
Band  1 


The  band  1  RF  AMP  is  an  externally  cooled  unit.  It  receives 
jamming  parameters  from  the  band  1  oscillator.  These  parameters 
are  modulated,  amplified,  and  then  transmitted. 

Band  2 

The  band  2  RF  AMP  is  also  an  externally  cooled  unit.  The  band 
2  RF  AMP  receives  jamming  parameters  from  the  band  2 
oscillator;  these  parameters  are  modulated,  amplified,  and 
then  transmitted. 


The  fault  locator  is  a  module  contained  in  the  band  1  and  band  2 
oscillators.  This  module  continuously  or  on  demand  tests  the 
band  1  or  band  2  osciallator  and  the  RF  AMP.  This  fault  locator 
module  performs  a  built-in  test  (BIT)  for  the  controlled  oscillator 
and  the  RF  AMP  LRUs. 

I.RU-  3 

The  low  band  receiver/processor  detects  and  processes  low  band 
transmissions  and  analyzes  them  in  conjunction  with  high  band 
transmissions  to  determine  all  threat  parameters.  It  uses  the 
computer  within  the  low  band  receiver  processor. 


The  purpose  of  the  test  requirements  document  analysis  was  to  determine 
the  types  and  complexity  of  the  tests  and  the  structure  of  the  TRD.  Three 
specific  features  of  the  TRD  tests  were  tabulated. 
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1)  Simple  tests  such  as  voltage,  current,  or  resistance  measurements 
were  counted.  These  types  of  tests  are  of  interest  because  of 
their  easy  implementation  on  the  UUT  simulator. 

2)  Non-simple  tests  were  rated  on  a  scale  from  1  to  10  (the  most 
complex  was  rated  as  a  1C).  This  was  done  with  the  intention  of 
having  the  scale  correspond  to  the  level  of  difficulty  of  writing  a 
section  of  code  for  a  specific  test. 

3)  The  number  of  high-power  tests  was  also  determined.  During 
these  tests  the  UUT  must  sink  or  source  more  than  1  watt  of 
power.  This  is  of  importance  in  regard  to  a  simulator,  because 
the  simulator  would  need  special  generators  and  heavy  duty 
internal  circuitry  to  support  high  power  tests. 

The  structure  of  the  TRD  was  examined  by  looking  at  the  following 
characteristics: 

1  )  The  percentage  of  the  total  tests  used  for  fault  isolation  was 
obtained.  This  is  of  significance  because  the  simulator  would 
be  helpful  in  checking  fault-isolation  paths. 

2)  The  number  of  entry  points  into  the  fault  isolation  section  of  the 
TRD  was  also  determined.  This  is  important  because  the  use  of 
a  simulator  would  aid  in  checking  these  paths. 

3)  The  number  of  branches  in  the  normal  test  part  of  the  TRD  was 
also  determined.  This  is  useful  in  establishing  how  much  a 
simulator  would  benefit  branch  checking. 


78 


r 


The  complexity  of  the  stimuli  and  response  was  estimated  separately, 
and  they  are  often  different.  Stimuli  of  a  constant  voltage  or  resistance 
were  not  classified  as  complex  stimuli;  likewise,  a  constant  voltage  or 
resistance  measurement  was  not  classified  as  a  complex  response.  The 
basic  idea  behind  the  complexity  levels  is  that  they  represent  how  difficult 
the  signals  would  be  to  produce.  What  follows  is  a  brief  definition  of  the 
levels  by  example. 

1.  Stimuli 

a.  A  step  function  with  no  rise  time  or  duration  specified 

Response 

a.  Measurement  of  two  discrete  ievels,  one  before  the  step 
function  and  one  after 

Each  of  the  specific  levels  has  an  upper  and  lower  limit. 

2.  Stimuli 

a,  A  pulse  of  specified  duration 

b.  A  step  function  with  step  occurring  at  a  specific  time  with 
respect  to  tile  test 

Response 

a.  Delay  measurement,  of  the  response  pulse  to  the  stimuli  pulse 

3.  Stimuli 

a.  Many  simple  or  level  one  tests 

b.  Ramp  function  with  a  specified  rise  time 
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c.  Simple  pulse  trains 

d.  A  feedback  loop  where  the  stimuli  voltage  is  changed  until 
the  response  voltage  changes  to  a  different  level 

Then  the  threshold  stimuli  voltage  is  recorded  as  the  response. 

Response 

a.  The  signal  pulse  will  remain  less  than  a  certain  value  upon 
application  of  the  stimuli  (i.  e. ,  surges  caused  by  application 
of  prime  power), 

b.  And  those  listed  in  stimuli  category. 

4.  Stimuli 

a.  Moderately  long  pulse  trains 
Response 

a.  Several  delay  tests 

b.  Moderately  long  pulse  trains 

5.  Stimuli 

a.  Long  pulse  trains 
Response 

a.  Long  pulse  trains 

6.  Stimuli 

a.  Complex  pulse  trains 
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Response 


a.  Complex  pulse  trains 

7.  Stimuli 

a.  No  stimuli  found  that  have  reached  levels  of  7  or  above 

Response  * 

a.  Complex  waveforms  with  rise  and  fall  times,  pulse  width 
limits,  and  lower  and  upper  limits  on  voltage  at  quiescent 
levels  specified 

8.  Stimuli 
a.  None 

Response 

a.  Several  responses  with  complex  waveforms 

9.  Stimuli 
a.  None 

Response 

a.  A  pulse  that  must  fit  inside  an  envelope 

This  would  give  the  waveform  upper  and  lower  limits  on 
the  period,  rise  and  fall  times,  and  voltage  at  all  points  on 
the  waveform. 

The  TRD  contains  a  description  of  the  terminals  that  are  on  the  UUT. 
Input,  output,  and  test  connections  were  counted  and  recorded  on  the  TRD 
analysis  table  (Table  3). 
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TABLE  3.  TRD  ANALYSIS  RESULTS 


TRD 

LVPS 

094  007326 

Rf  AMP 

094  007327 

FL 

094  007325 

LRU  2* 

UUT  Interface 

Number  Connections 

Total 

62 

147 

210 

- 

In 

17 

24 

81 

- 

Out 

36 

19 

76 

- 

Test 

35 

88 

60 

- 

TRD  Structure 

Number  Tests 

Total 

181 

406 

431 

528 

Normal 

62 

116 

96 

349 

fault  Isolation 

119 

290 

335 

179 

Maintenance  Actions 

Normal  Termination 

1 

4 

2 

2 

fault  Isolation 

49 

92 

97 

0 

Number  Branches 

54 

93 

94 

- 

Entry  Points 

27 

33 

49 

- 

Types  of  Test 

Resistance 

25 

6 

72 

55 

Voltage,  Current,  Power 

44 

103 

70 

301 

Complex  Stimul i 

Number  Steps 

38 

SO 

73 

- 

Average  Complexity 

1.8 

1.7 

2.2 

- 

Complexi ty  Measure 

68 

85 

158 

- 

Complex  Responses 

Number  Steps 

28 

72 

84 

153 

Average  Complexity 

1.6 

2.3 

2.7 

1 . 7 

Complexity  Measure 

44 

167 

228 

256 

High  Power  Tests 

UUT  Sink 

81 

177 

150 

- 

!JUT  Source 

81 

i 

3 

0 

* 

*  Data  from  LRU  2  was  taken  from 
a  TRD  Summary . 
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The  number  of  tests  in  both  normal  test  and  fault  isolation  sections  was 
tabulated.  This  portrays  the  length  of  the  TRD. 

A  maintenance  action  is  usually  some  physical  action  performed  on  the 
UUT  or  the  ATE.  The  maintenance  actions  in  both  the  normal  test  and 
the  fault  isolation  sections  were  counted.  Because  this  represents  some¬ 
thing  a  simulator  would  have  difficulty  performing,  this  is  relevant 
material  to  consider. 

The  number  of  branches  out  of  the  normal  test  section  was  counted.  Entry 
points  into  the  fault  isolation  section  were  also  counted.  This  information 
is  useful  in  determining  the  logic  flow  of  the  TRD. 

Resistance  measurements  and  voltage  and  current  measurements  were 
counted.  These  are  categorized  as  simple  tests.  Tests  that  were  more 
complex  than  simple  tests  were  rated  according  to  the  scale  as  discussed 
previously  in  this  section.  The  number  of  complex  steps  for  both  stimuli 
and  response  was  recorded.  The  complexity  levels  of  all  the  steps  for 
stimuli  and  response  were  summed  and  this  information  was  recorded  as  the 
complexity  measure.  This  was  done  separately  for  stimuli  and  response. 
Then  the  average  complexity  for  stimuli  and  response  was  determined. 

This  information  was  found  by  dividing  the  complexity  measure  by  the 
number  of  steps.  Once  again  this  was  done  for  both  stimuli  and  response. 

High-power  tests  that  cause  the  UUT  to  sink  or  source  high  power  are 
important.  A  simulator  design  would  be  altered  significantly  by  the 
presence  of  many  high-power  tests.  Therefore,  the  number  of  tests  where 
the  UUT  must  sink  or  source  high  power  was  recorded. 


The  following  are  the  conclusions  from  the  TRD  analysis  activity: 

•  The  average  complexity  levels  of  the  response  were  higher  than 
those  of  the  stimuli.  This  implies  that  a  simulator  would  be 
different  from  an  ATE.  The  simulator  would  have  more  sophis¬ 
ticated  signal  sources  and  generators  than  an  ATE,  but  it  would 
not  require  the  sensitive  sensors  and  measurement  devices  that 
an  ATE  has. 

•  Approximately  2/3  of  the  TRD  was  found  to  be  composed  of  simple 
tests.  A  simulator  that  would  handle  just  simple  tests  might  prove 
to  be  cost-effective  because  of  this  finding. 

•  In  the  TRDs  analyzed,  roughly  2/3  of  the  total  number  of  tests 
are  used  for  fault  isolation.  This  portrays  the  importance  of 
having  a  simulator  capable  of  checking  the  numerous  branches 
that  occur  in  fault  isolation. 

•  Assumptions  can  be  made  relating  the  physical  size  of  the  TRD 
to  its  complexity  levels  and  total  number  of  tests. 

1.  Complexity  levels  of  the  response  were  found  to  correspond 
to  the  thickness  of  the  TRD.  The  thicker  TRDs  were  found 
to  have  higher  response  complexity  levels. 

2.  A  correlation  between  the  thickness  of  the  TRD  and  the  total 
number  of  tests  was  found.  The  number  of  tests  in  a  TRD 
increased  with  increasing  TRD  thickness. 


ATLAS  Program  Analysis 


Selected  ATLAS  test  programs  from  the  F-15  TITE  project  were  analyzed. 
The  analysis  was  performed  with  the  intention  of  observing  qualities  that 
would  be  useful  in  establishing  the  baseline  requirements  for  the  UUT 
simulator.  The  analysis  consisted  of  looking  at  the  following  character¬ 
istics  of  a  test  sequence: 

1.  The  logic  flow  of  selected  tests  was  plotted.  This  was  done  to 
determine  the  importance  of  having  a  simulator  capable  of  forcing 
control  into  the  various  branches  of  the  test  program. 

2.  Occurrences  of  operator  messages  and  operator  actions  were 
tabulated.  The  operator  is  essentially  another  interface  between 
the  ATE  and  the  UUT.  If  the  operator  must  perform  a  substantial 
amount  of  communication  between  the  ATE  and  the  UUT,  that 
would  require  a  simulator  to  have  more  sophisticated  operator 
I/O  capabilities. 

3.  For  a  given  test  the  number  of  statements  was  counted.  This 
number  was  then  compared  to  the  complexity  level  established 
in  the  TRD  analysis.  This  would  provide  information  on  how 
accurate  the  TRD  stimulus  and  response  complexity  analysis  was. 
Test  programs  that  correspond  to  some  of  the  analyzed  TRDs 
were  used  for  test  program  analysis.  This  was  done  to  allow 
comparison  between  results  of  TRD  analysis  and  the  test  program 
analysis.  Also,  because  familiarity  with  a  group  of  UUTs  was 
obtained  during  the  TRD  analysis,  the  test  program  analysis  was 
aided  by  choosing  test  programs  for  the  same  UUTs.  The  specific 
TITE  LRUs  tests  which  were  analyzed  can  be  found  in  Table  4. 
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TABLE  4.  ATLAS  TEST  PROGRAM  LOGIC  FLOW  DIAGRAMS 


FL 

LVPS 

LRU-3 

RF  AMP 

TP  47 

TP  48 

T4  T909 

TP  83 

TP  32 

T4  T1926.  20 

TP  56 

TP  56 

T2  T9  (465-467)* 

TP  38 

TP  17 

T2  T1 

TP  13 

T3  T802 

TD  22.4 

T4  T947 

TP  71 

T2  T9  (310)* 

TP  23.  2 

T1  T12 

TP  37 

T3  T883 

TP  39 

Tests  in  the  test  program  are  usually  organized  and  identified  in  the  same 
way  as  they  are  in  the  TRD.  The  test  identifiers  are  TP,  which  is  short 
for  test  performance  (also  known  as  normal  test),  T  for  test,  and  TD, 
which  stands  for  test  diagnostics  or  fault  isolation.  The  FL,  LVPS,  and 
and  RF  amp  tests  are  identified  in  their  test  program  by  the  same  identi¬ 
fication  given  to  the  corresponding  test  in  the  TRD.  Tests  in  the  LRU-3 
have  a  more  complicated  test  identification  code  than  was  given  to  them 
in  the  TRD.  This  is  due  to  the  enormous  size  of  the  test  program.  The 
test  program  had  to  be  divided  into  five  segments  to  be  stored  on  the  disk 
file.  The  format  used  to  identify  a  test  is  of  the  form  TX  TY  (NNN),  where 
TX  is  the  segment  indicator,  and  TY  is  the  test  identification  taken  from 
the  TRD.  If  (NNN)  is  also  present,  it  is  a  further  breakdown  of  test 
TX  TY.  This  breakdown,  NNN,  along  with  TX  is  not  used  to  identify  a 
test  in  the  TRD;  only  TY  is  used  for  identification. 
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Appendix  B  contains  specific  examples  of  stimuli  and  response  classifications. 
The  examples  make  references  to  specific  UUT  pins,  signal  values,  and 
measurement  units.  *  This  was  done  to  allow  the  reader  to  visualize  the 
amount  of  information  in  the  TRD. 

Appendix  C  contains  flow  diagrams  for  some  of  the  test  program  sections. 

The  diagrams  are  classified  by  the  acronym  for  the  LRU  and  by  the  test 
number.  In  some  cases  the  complexity  of  the  stimuli  and  responses  as 
determined  from  the  TRD  is  also  included.  The  straight-line  code  segments 
have  been  abbreviated,  and  a  short  word  is  given  to  present  a  rough 
indication  of  what  the  test  program  is  doing.  The  important  information  to 
be  conveyed  in  the  flow  diagrams  is  the  amount  and  structure  of  the  branching. 


Results  from  the  logic  flow  diagrams  show  that  the  tests  are  usually 
shallow,  serial  code.  Several  other  more  specific  details  about  program 
organization  and  structure  were  observed. 

1.  No-goes  are  .usually  directed  to  fault  isolation,  or  occasionally 
they  are  tested  further  to  determine  which  entry  points  into 
fault  isolation  is  correct. 

2.  Subroutines  are  seldom  called  during  a  normal  test.  In  fault 
isolation,  subroutines  are  used  to  obtain  setup  procedures  or 
operator  messages  and  manual  intervention. 

3.  There  was  found  to  be  a  high  level  of  correspondence  between  the 
TRD  test  number  and  the  actual  code  sequence  in  the  test  program. 
Tests  occurred  in  the  same  order  in  the  test  program  as  they  did 
in  the  TRD. 
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The  number  of  statements  for  a  test  was  found  to  agree  well  with  the 
complexity  levels  for  {he  stimulus  and  response  established  in  the  TRD 
analysis. 


Operator  messages  and  manual  interventions  occurred  on  5  to  10%  of  the 
normal  tests,  but  appeared  much  more  frequently  in  fault  isolation.  It  was 
difficult  to  determine  an  exact  figure  because  of  the  use  of  subroutines  to 
perform  operator  messages  and  actions.  An  estimated  50  to  75%  of  the 
tests  in  fault  isolation  performed  one  or  more  operator  message  or  action. 


s 
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SECTION  4 


SOLUTION  ALTERNATIVES 


This  chapter  presents  a  bottom-up  view  of  the  alternate  solutions  with 
which  a  UUT  simulator  could  be  realized.  The  presentation  is  structured 
to  proceed  from  the  applicable  techniques  and  technologies,  through  the 
trade-offs,  and  into  the  recommended  solution.  The  following  summarize 
this  progression: 

•  Simulation  Techniques:  This  part  of  the  chapter  presents 
summaries  of  simulation  techniques  documented  in  the  literature, 
which  are  potentially  applicable  to  a  UUT  simulator. 

•  Applicable  Technologies:  A  list  of  technologies  and  their 
applicability  and  implication  on  a  UUT  simulator  are  presented. 
The  objective  is  to  constrain  the  set  of  solutions  considered  for 
a  UUT  simulator  to  the  technologies  which  will  be  available  and 
proven  by  the  time  of  implementation  of  the  UUT  simulator. 

•  Solution  Components:  Based  on  the  requirements  in  Section  3,  the 
simulation  techniques  discussed  above  and  the  technology  con¬ 
straints,  specific  UUT  simulator  components  are  identified.  The 
solutions  are  then  classified  into  a  smaller  set  to  permit  them 

to  be  used  in  a  trade-off. 

•  Trade-off  Descriptions:  This  section  of  the  chapter  contains  a 
discussion  of  the  trade-off  process  and  results  in  choosing  a 
specific  solution  as  the  recommended  UUT  simulator  approach 
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for  subsequent  refinement.  A  discussion  is  also  included  on  the 
sensitivity  of  the  choice  to  some  of  the  key  factors  entering  the 
trade-off. 


SIMULATION  TECHNIQUES 

Computer  simulation  of  electrical  circuits  has  become  a  very  important 
tool  in  the  area  of  computer-aided  design  (CAD),  automatic  test  equipment, 
and  test  program  generation.  Although  each  of  these  application  areas 
may  require  some  form  of  additional  features  such  as  fault  insertion  and 
fault  isolation,  the  basic  simulation  capability  required  remains  very 
much  similar.  In  this  section,  the  state  of  the  art  in  a  circuit  simulation 
system  is  reviewed  and  assessed. 

Simulation  techniques  to  be  used  in  the  UUT  simulator  were  found  from 
documentation  of  existing  techniques  and  from  extensions  derived  by 
considering  specific  characteristics  of  the  application  of  a  UUT  simulator 
oriented  primarily  to  support  TPS  verification  and  validation. 

The  approach  to  describing  these  techniques  here  includes  the  following: 

•  Describe  a  set  of  categories  which  will  help  to  differentiate  the 
simulation  techniques. 

•  Present  more  detailed  descriptions  of  some  of  the  more  promising 
techniques  in  each  category. 

•  Attach  summaries  of  the  references  from  the  literature. 
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•  Discuss  the  feasibility  and  appropriateness  of  considering 
specific  techniques  for  use  in  a  UUT  simulator  oriented  to 
support  TPS  verification  and  validation. 

In  general,  the  simulation  techniques  currently  used  can  be  divided  into 
five  categories: 

•  Circuit  Level  Analog  Simulation 

•  Digital  Circuit  Gate- Level  Simulation 

•  Functional  Simulation 

•  Interface- Level  Simulation 

•  Mixed- Level  Simulation 

A  survey  of  these  techniques  was  conducted  and  is  reported  in  the  following 
subsections. 

Circuit  Level  Analog  Simulation 

In  this  approach,  the  simulation  is  performed  down  to  the  individual 
circuit  element  component  levels,  such  as  resistor,  capacitor,  and 
transistor.  As  such,  it  can  provide  very  accurate  simulation  with  regard 
to  signal  amplitudes  and  timing  analysis.  This  is  particularly  useful  in 
performing  circuit  analysis  in  circuit  design.  Both  analog  as  well  as 
digital  circuits  can  be  simulated  at  this  level.  However,  the  number  of 
computations  involved  could  be  overwhelming  because  elaborate  circuit 
modeling  algorithms  have  to  be  implemented.  This  technique  is  best 
suited  for  analog  circuits.  For  large  complex  circuits,  such  as  digital 
LSICs  and  VLSICs,  the  computation  can  get  out  of  hand  rapidly.  In 
those  cases,  simulation  programs  at  a  higher  level  may  be  more  efficient. 
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Currently  there  are  more  than  30  simulation  packages  that  are  available 
for  circuit  simulation  at  this  level.  Some  of  them  are  specialized  packages 
that  are  used  for  special  applications  only.  Together  they  can  provide 
features  such  as  statistical  or  tolerance  analysis,  sensitivity,  and  stability 
analysis,  as  well  as  DC,  AC,  and  transient  analysis.  Some  of  the  more 
prominent  simulation  systems  that  have  been  developed  are  shown  in 
Tables  5  through  8,  together  with  their  characteristics  and  capabilities. 

The  contents  of  these  tables  were  extracted  from  "Computer-Aided 
Design,  "  IEEE  Spectrum,  October  1975.  To  provide  a  better  insight  into 
the  nature  of  a  circuit  level  analog  simulation  system,  one  of  the  program 
packages  is  examined  in  more  detail— the  SPICE  2  system. 

SPICE  2  is  a  general-purpose  digital  computer  program  to  simulate  the 
electrical  performance  of  semiconductor  circuits.  It  was  developed  by 
Dr.  Lawrence  Nagel  of  the  Electronics  Research  Lab. ,  University  of 
California,  Berkeley. 

The  program  can  be  used  to  simulate  most  electronic  circuits  to  determine 
the  quiescient  operating  point,  the  time-domain  response,  or  the  small- 
signal  frequency-domain  response  of  the  circuits.  It  provides  for  12 
types  of  built-in  device  models  for  the  common  circuit  components. 

These  12  circuit  elements  include  the  following: 

•  Linear  Elements 
Resistor 
Capacitor 
Inductor 

Coupled  Inductors 


TABLE  5.  MAJOR  COMPUTER-AIDED  CIRCUIT  DESIGN  PROGRAMS 
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I; 

i 

i  i 


Commercially  available:  1  Classified; 

Handling  charge:  2  No  Charge 


TABLE  6.  PROGRAM  FEATURES 


SNAF’  I  Symbolic  network  function 


TABLE  7.  ANALYSIS  DETAILS 
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SUPER  SCEPTRE 


TABLE  8.  OUTPUT  DETAILS 


Independent  Voltage  Source 
Independent  Current  Source 
Linear  Voltage-Controlled  Current  Source 

•  Nonlinear  Elements 

Nonlinear  Voltage -Controlled  Current  Source 
Diode 

Bipolar  Junction  Transistor  (BJT) 

Junction  Field- Effect  Transistor  (JFET) 

Insulated- Gate  Field-Effect  Transistor  (MOSFET) 

Each  model  in  turn  contains  many  parameters  which  are  used  to  reflect  the 
individual  device  characteristics  and  the  technologies  used  in  the  chip 
manufacturing.  These  parameter  values  should  be  updated  periodically  to 
reflect  the  current  state  of  IC  technology.  As  emerging  technologies  such 
as  VMOS  and  SOS  are  adopted,  entirely  new  model  types  will  have  to  be 
incorporated  into  the  package. 

For  performing  the  circuit  analysis,  the  program  uses  a  modified  nodal 
analysis  method  which  generates  a  well- conditioned  system  of  equations 
that  can  be  solved  with  a  minimal  amount  of  computational  effort.  The 
program  can  handle  linear  as  well  as  nonlinear  networks.  By  using  the 
nodal  analysis  method,  the  system  of  algebraic- differential  equations 
describing  the  complete  circuit  can  be  reduced  to  one  of  the  three  types  of 
systems: 

•  System  of  Linear  Equations 

•  System  of  Nonlinear  Equations 

•  System  of  Ordinary  Differential  Equations 
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Of  the  many  numerical  analysis  algorithms  available  for  each  type  of  the 
system,  SPICE  2  uses  a  sparse  matrix  technique,  a  modified  Newton- 
Raphson  algorithm  (the  simple- limiting  method  of  Colon),  and  the  trapezoidal 
rule  integration  algorithm,  respectively,  to  solve  the  three  types  of  systems. 
A  modified  Gear's  algorithm  is  also  optionally  available  to  solve  the  third 
type  of  systems. 

In  all,  a  combination  of  ten  analysis  types  are  available  in  the  SPICE  2 
program  to  perform  the  simulation  of  an  electronic  circuit.  They  include 
the  following: 

•  DC  Analysis 

DC  Operating  Point 

Linearized  Device  Model  Parameterization 
Small-Signal  Transfer  Function 
DC  Transfer  Curves 
Small- Signal  Sensitivities 

•  AC  Analysis 

Small-Signal  Frequency-Domain  Response 
Noise  Analysis 
Distortion  Analysis 

•  Transient  Analysis 

Time- Domain  Response 
Fourier  Analysis 


The  SPICE  2  system  has  approximately  15,000  source  statements  written 
in  FORTRAN  IV  to  run  on  the  CDC  6400  computer.  However,  the  program 
has  also  been  adapted  for  use  on  the  IBM  and  Honeywell  computer  systems. 


Although  this  circuit  simulator  system  is  considered  to  be  one  of  the 
industry  standards,  it  has  several  shortcomings  that  need  to  be  improved. 
Additional  capabilities  that  would  be  desirable  include: 

•  Model  flexibility 

•  User  specified  functions 

•  Worst- case  analysis 

•  Monte  Carlo  (statistical)  analysis 

•  Radiation  effect  analysis 

As  mentioned  before,  most  of  the  simulation  systems  described  so  far 
have  been  developed  for  circuit  analysis  in  design  automation.  For  direct 
application  in  a  UUT  simulator  modifications  and  additional  features  will  be 
required. 

Note  also  that  simulation  of  a  discrete  analog  circuit  can  always  be 
carried  out,  using  circuit  breadboards  and  analog  computers.  For 
integrated  circuits,  the  presence  of  parasitic  components  makes  this 
traditional  method  of  measuring  electrical  characteristics  of  a  circuit 
unacceptable. 
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Digital  Circuit  Gate- Level  Simulation 

For  a  digital  circuit,  a  commonly  used  technique  is  to  model  the  actual 
circuit  as  a  connection  of  gates  (AND,  NAND,  OR,  NOR)  and  perform  the 
simulation  at  this  gate  component  level.  This  technique  typically  can 
provide  accurate  simulation  to  a  detailed  circuit  level  with  respect  to  logic 
states  and  timing  analysis.  Thus,  this  technique  is  often  used  in  digital 
test  program  generation  systems  where  detailed  simulation  capability  is 
required. 

Most  simulators  model  at  least  three  logic  states  (logic  0,  logic  1,  unknown 
x)  and  use  a  unit  delay  timing  model.  For  certain  applications  a  higher 
accuracy  is  needed,  requiring  more  logic  states  (such  as  high  impedance 
•2E,  rising  edge  r,  fall  edge  i ,  etc.  )  or  better  timing  accuracy. 


Systems  that  utilize  this  simulation  technique  include: 


D4-LASAR 

CAFIG 

FAIRTEST 

FANSIM 


(Teradyne) 

(Bendix  Corporation) 
(Boeing  Computer  Services) 
(GTE  Laboratories) 


In  the  D4-LASAR  system,  the  only  building  block  used  is  a  NAND  gate; 
i.  e. ,  all  circuit  elements  are  represented  hy  NAND  gates  only.  With 
this  modeling  technique,  it  is  algorithmically  simple  to  implement  fault 
simulation  and  the  path  sensitization  method  in  vector  generation.  In 
addition,  it  enables  the  internal  operation  of  an  IC  to  be  evaluated  at  the 
detailed  NAND  level  during  fault  simulation. 
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For  the  other  three  systems,  instead  of  using  the  NAND  function  only, 
they  allow  a  gate  to  be  any  of  several  Boolean  functions  with  multi-inputs 
and  single  output.  They  also  enjoy  the  similar  advantages  of  algorithmic 
simplicity  and  detailed  fault  analysis. 

The  complexity  of  a  circuit  is  generally  measured  in  terms  of  "equivalent 
gate  counts;"  that  is,  a  circuit  is  broken  down  into  the  basic  building 
gates,  such  as  NAND,  NOR,  and  OR  gates,  and  the  count  of  all  of  these 
gates  then  determines  the  size  and  complexity  of  the  circuit. 

A  circuit  is  represented  as  a  logic  diagram  showing  the  interconnection 
of  ICs.  The  coding  of  such  a  logic  diagram  can  be  fairly  simple,  requiring 
mainly  specifications  of  the  IC  types  and  the  interconnection  of  various 
IC  and  edge  connector  pins.  Other  systems  such  as  D4-LASAR,  however, 
would  require  a  good  understanding  of  digital  logic  and  the  simulation 
system  in  order  to  correctly  model  a  circuit.  This  requirement  is  due  to 
circuit  conditions  which  cannot  be  modeled  by  the  system  and  which  call 
for  a  "workaround"  technique,  substituting  the  original  circuit  with 
special  circuit  elements. 

A  brief  description  of  these  four  systems  which  use  the  gate-level 
simulation  technique  is  given  below: 

D4-LASAR  is  one  of  the  most  commonly  used  automatic  test  generation 
systems  in  the  industry.  It  was  first  developed  in  1968.  The  basic 
principle  behind  D4-LASAR  is  for  all  circuits  to  be  modeled  in  terms  of 
NAND  gates.  The  reasoning  is  that  it  is  more  efficient  to  run  a  system 
optimized  for  one  device  type  rather  than  for  one  which  has  special  codes 
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for  many  different  types  of  devices.  Also  it  is  argued  that  a  detailed 
NAND  model  of  an  IC  is  the  only  effective  way  to  test  out  the  IC's  internal 
functions,  because  it  allows  for  the  analysis  of  stuck  faults  at  the  gate 
rather  than  at  the  IC  level,  and  that  a  worst- case  timing  analysis  is  better 
carried  out  with  a  NAND  gate  model. 

The  main  drawback  of  this  system  is  that  it  is  computationally  inefficient. 
Besides,  the  user  skill  level  must  be  relatively  high  in  order  to  recognize 
the  need  for  special  models  in  many  cases. 

The  D4-LASAR  system  is  maintained  by  Teradyne,  Inc. 

CAFIG  is  a  simple  simulation  system  developed  by  Bendix  Corporation 
mainly  for  in-house  use.  As  such,  it  is  not  meant  to  serve  as  a  general- 
purpose  test  programming  system.  Some  of  the  restrictions  on  the 
system's  capability  are  as  follows: 

•  1000  gate  limit  in  circuit  complexity 

•  No  bus  pin  concept 

•  Manual  wired  nodes 

•  No  one  shot 

•  Slow  fault  simulation 

•  No  interactive  faulting  and  display  capability 
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The  FAIRTEST  system  is  maintained  by  Boeing  Computer  Services  which 
obtained  the  software  from  Fairchild  Systems  Technology. 

The  FAIRTEST  system  consists  basically  of  three  major  portions:  the 
good  circuit  simulator  (FAIRSIM),  the  stimuli  generator  (FAIRGEN),  and 
the  fault  simulator.  The  system  is  rated  as  an  excellent  good-circuit 
simulation  capability.  In  particular,  the  simulation  command  language 
is  very  powerful,  including  conditional  language  commands.  The  good- 
circuit  simulation  can  be  carried  out  over  three  states  with  assignable 
delays  (0  through  64)  for  transitions  high  and  transitions  low.  There  are 
altogether  12  primitives  which  can  be  used  in  modeling  a  circuit  for 
FAIRSIM. 

The  fault  simulator  can  simulate  255  faults  in  parallel.  Faults  included 
in  the  analysis  are  stuck  faults,  both  external  and  internal  to  an  IC,  and 
shorts  between  adjacent  IC  pins. 

FANSIM  was  developed  by  GTE  Laboratories  for  use  within  GTE  for  design 
simulation,  test  verification,  and  automatic  test  generation  of  digital 
circuits.  The  system  is  well-known  for  its  concurrent  fault  simulation. 

The  technical  features  in  the  good- circuit  simulator  include  a  nanosecond 
delay  (up  to  16,000)  simulation  capability  in  which  two  delays  are 
assigned  to  each  output  of  an  element.  The  simulator  also  uses  an  inertial 
delay  model  for  ICs  to  filter  out  multiple  transitions  which  occur  within 
the  delay  time  of  a  device.  Large  pulses  are  flagged  via  special 
messages. 
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Using  the  concurrent  fault  simulation  technique  which  GTE  developed, 
the  system  is  quite  computationally  efficient.  To  overcome  the  large 
memory  requirements  inherent  in  this  approach,  a  hardware-based 
virtual  memory  system  is  used. 

The  IC  and  circuit  modeling  is  generally  convenient  except  for  bus- 
structured  circuits  because  there  is  no  concept  of  an  IC  bus  pin. 

The  characteristics  and  capabilities  of  these  four  systems  can  be 
summarized  in  Tables  9  through  14. 

In  general,  the  gate-level  simulation  approach  seems  to  have  the  following 
advantages: 

•  The  simulation  does  conform  closely  to  the  actual  implementation. 
Thus,  it  provides  a  relatively  accurate  model  from  the  stand¬ 
point  of  logic  and  timing. 

•  When  used  as  a  tool  in  automatic  testing  and  timing  isolation, 
the  diagnosis  can  be  directly  traced  to  the  failing  logic  gates. 

•  Single  fault  modelings  and  test  development  algorithms  using 
this  technique  have  been  well- developed  over  the  years  and  are 
relatively  simple  to  implement. 

•  A  great  deal  of  experience  on  this  approach  has  been  accumulated 
by  test  designers  together  with  much  literature  and  software 
support. 
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TABLE  9.  SYSTEM  CAPABILITY  (GATE  LEVEL) 


TABLE  10.  GENERAL  CONSIDERATIONS  (GATE  LEVEL) 


TABLE  12.  CIRCUIT  MODELING  (GATE  LEVEL) 


TABLE  14.  FAULT  SIMULATION  (GATE  LEVEL) 


However,  there  are  also  some  serious  disadvantages,  especially  with  the 
advent  of  high  density  integrated  circuit  chips. 


•  A  gate  equivalent  description  of  the  device  is  not  always  found 
in  the  data  sheet.  Alternative  descriptions  derived  from  the 
specifications  may  be  incomplete  and  are  difficult  to  use  for 
detailed  analysis. 

•  Gate  delays  are  not  normally  given  in  the  description.  In  order 
to  obtain  accurate  timing  analysis,  gate  delays  will  have  to  be 
assigned  correctly,  based  on  additional  information. 

•  Model  preparation  is  a  difficult  and  laborious  process.  Informa¬ 
tion  on  signal  and  gates  internal  to  the  device  model  has  to  be 
stored.  Internal  signal  transitions  have  to  be  handled.  All  of 
these  consume  both  time  and  memory. 

•  Depending  on  the  circuit  complexity,  this  approach  may 
require  a  large  volume  of  details  to  be  simulated. 

Functional  Simulation 

As  the  circuit  size  increases  with  the  development  of  LSI  and  VLSI  chips, 
the  amount  of  simulation  to  be  performed  at  the  gate  level  tends  to  become 
overwhelming.  In  many  cases,  the  gate  level  simulation  often  offers  too 
much  resolution  and  provides  too  much  irrelevant  data.  The  gate  level 
approach  becomes  computationally  inefficient  and  costly.  A  single  chip 
may  now  exceed  the  entire  capacity  of  some  ATG  systems  if  gate-level 
modeling  is  used.  A  transition  to  higher  levels  of  simulation  becomes 
necessary  to  reduce  the  number  of  details  that  need  to  be  analyzed,  to 
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simplify  the  model  specifications,  and  to  provide  the  needed  flexibility  to 
cope  with  technological  changes.  The  functional  simulation  has  become 
the  more  popular  approach. 

The  functional  simulation  may  be  regarded  as  a  pseudo  gate-level  approach 
in  which  the  complexity  of  the  basic  building  blocks  has  been  increased 
from  simple  gates  to  complex  functions.  The  complexity  of  these  functional 
blocks  often  depends  on  the  partitioning  of  the  circuit  functions  because  of 
packaging  constraints.  They  may  vary  from  simple  functionals  such  as 
gates  and  flip-flops,  to  medium  complex  functionals  such  as  RAMs, 

ROMs,  shift  registers,  etc. ,  to  complex  functionals  such  as  ALUs,  bit- 
sliced  microprocessors,  PIAs,  etc.  Thus  typically  SSI,  MSI,  and  even 
LSI  circuit  packages  may  become  the  functional  block.  The  more 
complex  the  basic  functional  block  is,  the  greater  the  gain  in  computational 
efficiency.  As  the  building  block  components  achieve  greater  functional 
complexity,  the  need  for  simulating  their  internal  workings  diminishes. 

This  in  turn  enables  the  functional  simulation  approach  to  become  more 
desirable. 

Simulation  algorithms  are  used  for  these  complex  gates  to  produce  the 
appropriate  response  on  the  gate  outputs  for  each  of  the  different  input 
stimuli.  There  are  several  approaches  in  preparing  these  functional 
models.  A  functional  model  has  inputs,  internal  state  variables,  and 
outputs.  The  response  is  then  determined  by  the  modeling  algorithm. 


One  approach  involves  using  a  subroutine  to  solve  logic  equations  written 
for  the  specific  functional  block.  The  subroutine  will,  in  general,  describe 
a  logical  function  of  combinational  or  sequential  nature.  Given  the  values 
for  an  input  stimuli  vector  and  the  current  internal  states,  the  output 
response  can  be  computed.  This  is  shown  in  Figure  3. 

The  timing  relationships,  such  as  the  propagation  delays  (D..),  which 
depend  on  the  way  the  signals  propagate  from  inputs  to  outputs  in  the  model, 
have  to  be  considered  and  taken  care  of.  With  this  approach,  the  modeling 
subroutine  can  be  written  with  the  level  of  accuracy  given  in  the  data  sheet 
with  regard  to  logic  and  timing  relations.  In  addition,  there  are  no 
internal  signals  or  gates  in  the  model  which  have  to  be  handled  other  than 
the  state  variables,  thereby  saving  memory  space  and  processing  timing. 

In  general,  these  modeling  programs  are  written  in  a  type  of  high-level 
language  to  directly  emulate  the  device  behavior. 

At  the  primitive  gate  level,  only  simple  single- level  Boolean  functions  are 
involved.  However,  at  the  complex  gate  level,  the  modeling  algorithms 
must  be  able  to  deal  with  multiple  levels  of  logic  and  may  also  be  required 
to  retain  and  use  memory  state  values. 

Another  approach  is  to  transpose  the  information  in  the  data  sheet  concerning 
the  logic  and  timing  relations  directly  into  a  function  table  and  then  perform 
a  table  look-up  operation  to  perform  the  simulation.  In  this  way,  the  input 
values  are  directly  or  indirectly  used  to  "address"  the  proper  output 
response.  In  general,  it  is  quite  easy  to  write  a  function  table  description 
because  most  logical  functions  for  a  digital  device  are  often  provided  in 
the  form  of  a  function  table. 
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INPUT 
X,  _ 


X 


M 


FUNCTIONAL 

MODEL 


OUTPUT 


INPUT  VECTOR  X  =  (X,,  X2. ...  XM) 
OUTPUT  VECTOR  Y  =  (Yv  Y2, ...  YN) 
INTERNAL  STATES  S  =  (S,,  S2, ...  SL> 


Figure  3.  Functional  Simulation  Representation 


In  modeling  a  device  with  the  function  table,  many  symbols  must  be 
allowed  for.  These  symbols  correspond  to  the  more  important  symbols 
found  in  data  sheets.  The  symbols  are  used  to  define  input  states  or 
input  state  transitions  and  to  trigger  new  events  for  the  outputs.  They 
include: 

L  Low 

H  High 

Z  High  Impedence— Tri-State 

UN  Unknown 

X  Irrelevant 

PP  Positive  Pulse 

NP  Negative  Pulse 

t  Positive  Going  Edge 

i  Negative  Going  Edge 

NO...  No  Change  ... 

TO  Toggle 

UNL  Transition  Unknown  to  Low 

UNH  Transition  Unknown  to  High 

The  functional  simulation  approach  is  being  used  currently  by  many  test 
development  systems.  This  reflects  the  cost  and  difficulties  associated 
with  the  gate-level  approach  in  handling  high  density  integrated  circuits. 
In  .act,  the  FAIRTEST  and  F.ANSIM  systems  described  previously  in  the 
gate-level  simulation  section  can  actually  be  considered  to  have  a  partial 
functional  simulation  capability  with  the  use  of  several  complex  gates  as 
primitives. 
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From  the  literature,  25  test  development  and  design  aid  systems  have 
been  identified  to  use  the  functional  simulation  approach.  They  include 
the  following: 


AAI  Simulator 
AIDS 

CAI  Simulator 

CAPS  II 

DFA 

DIGISAT 

FAST 

FLASH 

F/ LOGIC 

GRASS 

INDICATES 

LAMP 

LOGCAP  II 

LOGICV 

LOGOS  III 

LSP 

LSTV 

MICROSIM 

NEWSIM 

SALOGS 

SIMSTRAN 

TASC 

TEGAS  II 

TESTAID  III 

TESTGEN 


(AAI  Corporation) 

(Applied  Technology  Inc.  ) 
(Computer  Automation) 

(Gen  Rad) 

(Hughes  Aircraft  Company) 

(Hughes  Aircraft  Company) 
(Honeywell  Information  Systems) 
(Micro  Systems  Division) 

(Bell  Northern  Research) 

(Raytheon  Company) 

(Sperry  Uni  vac) 

(Bell  Telephone  Laboratories) 
(National  CSS) 

(Westinghouse) 

(Grumman  Aerospace  Corporation) 
(Naval  Surface  Weapons  Center) 
(NCA  Corporation) 

(Instrumentation  Engineering) 
(Texas  Instruments) 

(Sandia  Laboratories) 

(Rockwell  International) 

(Pacific  Applied  Systems) 
(Comprehensive  Computing) 
(Hewlett  Packard) 

(Hughes  Ground  Systems) 
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The  capabilities  and  characteristics  are  summarized  in  Tables  15  through 

20. 

The  content  of  these  tables  is  extracted  from  "Selection  Guide  for  Digital 
Test  Program  Generation  Systems"  by  Henckels,  Haas  and  Brown,  Inc. 

Compared  to  gate-level  simulation,  it  usually  takes  a  longer  time  to  develop 
a  system  using  the  functional  simulation  approach,  because  it  requires 
more  advanced  programming  techniques,  more  sophisticated  considerations, 
and  more  development.  In  order  to  allow  complex  LSI  circuits  to  be 
modeled  as  functional  blocks,  some  form  of  special  high  level  functional 
modeling  language  or  circuit  description  language  is  required  to  achieve 
modeling  flexibility.  The  circuit  is  first  described  in  this  high  level 
language;  the  language  is  then  compiled  for,  or  interpreted  by,  the 
simulator.  In  any  event,  it  will  generally  require  a  highly  skilled  user  to 
accurately  transpose  the  logic  diagram  of  an  IC  into  a  flow  chart  and  then 
convert  it  into  software  program. 

There  are  other  additional  drawbacks  to  the  functional  approach.  The 
functional  simulation  is  only  an  approximation  to  the  actual  overall  design. 
There  are  fewer  timing  and  implementation  details  in  the  functional 
models.  Timing  paths  through  the  circuits  are  generally  less  accurately 
modeled  than  those  given  by  the  primitive  gate  level.  In  addition,  the 
fault  modeling  becomes  more  complex.  Because  most  internal  gate 
faults  are  not  modeled,  the  internal  stuck  fault  coverage  of  the  test 
generation  algorithm  may  not  be  adequate.  Diagnosis  data  generated  from 
such  analysis  may  not  cover  many  internal  package  failures.  Diagnosis 
of  the  failing  primitive  gate  may  no  longer  be  possible. 
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na  va  liable 


TABLE  15.  SYSTEM  CAPABILITY  (FUNCTIONAL)  (continued) 


Unavailable 


TABLE  15.  SYSTEM  CAPABILITY  (FUNCTIONAL)  (concluded) 


TABLE  16.  GENERAL  CONSIDERATIONS  (FUNCTIONAL)  (continued) 


TABLE  16.  GENERAL  CONSIDERATIONS  (FUNCTIONAL)  (concluded) 


TABLE  17.  IC  MODELING  (FUNCTIONAL) 


MICKOSIM  35  complex  300  No/Yi-s  Assembly  code  Circuit  language.  Microprocessor 

functionals'  ICs  and  Control  Moduli' 

primitives  specification 

language  (MCI  ) 


FAST  ICs  are  300  Yes/Yes  Truth  tables,  Special  purpose  None 

modeled  as  generated  high  level 

truth  tables  via  standard  language 


TABLE  17.  IC  MODELING  (FUNCTIONAL)  (continued) 


LSTV  30  complex  None  yet  Yes/Yes  FORTRAN  code.  Circuit  language,  PI  A  primitive,  also 

functionals  Boolean  equations  ICS  and  can  accept  binary  IC 

primitives  models  for 

’  proprirtary  ICs 


IC  MODELING  (FUNCTIONAL)  (concluded) 


private  library  as 
well  as  main  one. 


TABLE  18.  CIRCUIT  MODELING  (FUNCTIONAL)  (concluded) 


,  000  primitive.* 


GOC )I  > CIRCUIT  SIML!L  A  T  ION’  ( F U N C  T ION  A  U 


MIC'HOSIM  ( iood  0,  1  ,  /  Nominal  delay  on  Detects  .simultaneous  1  xcelli  nt,  powerful 

inc  remental  language  outputs  :  ransitions  on  KI  s  interactive  display 


TABLE  19.  GOOD-CIRC  UIT  SIMULATION  (FUNCTIONAL)  (continued) 


135 


incremental  language  propagation  race  detection  batch  orn-med 

incl.  cloe k  generators 


36 


TABLE  20.  FAULT  SIMULATION  (FUNCTIONAL)  (continued) 


Memory  safi,  sal 
Internal  K  faults 


TABLE  20.  FAULT  SIMULATION  (FUNCTIONAL)  (concluded) 


However  the  advantages  of  the  functional  approach  in  many  ways  outweigh 
its  disadvantages,  particularly  for  high  density  complex  circuits.  The  key 
is  in  computational  efficiency.  With  today's  IC  technology,  the  size  of  a 
UUT  modeled  in  terms  of  logic  gates  can  become  too  large.  Simulation 
of  a  complex  functional  block  may  be  slower  than  that  of  a  primitive  gate, 
but  it  is  still  much  faster  than  simulating  the  tens  of  thousands  of  primitive 
gates  that  might  otherwise  be  required  at  the  gate  level  approach.  The 
functional  modeling  approach  also  requires  less  storage  space  and  can 
handle  test  generation  on  functional  design  in  a  much  more  powerful  way 
for  high  density  chips  such  as  ROM,  RAM,  etc. 

The  other  advantages  include  the  following: 

•  The  functional  information  is  much  easier  to  obtain  from  the 
semiconductor  vendors  than  that  for  gate-level  models. 

•  Global  circuit  controls  are  easier  to  identify. 

•  There  is  a  large  reduction  in  the  volume  of  details  to  be  processed. 

•  Diagnosis  to  functional  subunits  is  possible. 

•  Some  analog  circuits  can  also  be  simulated. 

There  can  be  several  degrees  or  levels  of  functional  simulation,  ranging 
from  the  modeling  of  primitive  IC  gates  as  Boolean  functions  to  the 
abstract  description  of  the  total  circuit  function.  The  selection  of  the 
level  of  simulation  is  then  dependent  on  the  trade-off  of  storage  space  and 
execution  time  vs  accuracy  and  completeness. 
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With  the  development  of  LSI  and  VLSI  technology,  there  is  growing  support 
for  the  higher  level  approach,  also  known  as  behavioral  simulation. 

Interface-Level  Simulation 


The  interface- level  simulation  can  be  considered  as  a  subset  of  the 
functional  level  simulation,  whereby  a  table  look-up  technique  is  used  to 
implement  the  simulation.  This  approach  has  been  widely  used  in  areas 
such  as  trainer  simulation  systems.  By  storing  the  appx-opriate  output 
values  or  responses  for  each  possible  combination  of  inputs  in  a  set  of 
tables,  this  approach  can  provide  a  limited  simulation  capability  on  the 
behavior  of  the  UUT.  In  general  only  static,  typically  table-driven, 
input/output  relationships  are  assumed. 

A  table-driven  circuit  simulator  operates  upon  the  topology  of  the  circuit. 
The  circuit  is  stored  as  a  set  of  tables  indicating  information  such  as 
fan-in  list,  fan-out  list,  signal  or  logic  values,  types  of  gates,  etc.  For 
a  given  set  of  input  values,  a  set  of  output  values  can  be  determined,  which 
in  turn  will  become  the  inputs  for  the  next  table.  In  this  manner,  a 
simple  dynamic  simulation  system  can  be  implemented. 

Simple  fault  detection  and  fault  simulation  can  also  be  carried  out  by 
determining  the  outputs  for  a  sequence  of  input  signals.  The  output  values 
can  then  in  turn  be  used  in  conjunction  with  a  fault  signature  table  for 
fault  detection  or  simulation.  In  general,  some  form  of  special  purpose 
language  or  processing  will  be  required  to  support  these  additional  entries 
of  input/output  relationship  tables  into  the  simulation  system. 
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The  interface  level  of  simulation  cannot  be  used  to  provide  details  on 
implementation  and  timing  analysis.  However,  it  does  offer  the  following 
advantages: 

•  It  does  not  require  skilled  users. 

•  The  computer  resource  required  is  relatively  small. 

•  Fast  simulation  time  is  possible. 

Mixed-Level  Simulation 


With  the  advent  of  high  density  chips  such  as  VLSI  and  VHSI  circuits  and 
also  hybrid  (ana log/ digital)  systems,  a  new  approach  called  mixed-level 
(or  multi-level)  simulation  has  been  proposed,  whereby  simulations  can 
be  performed  at  any  one  of  the  following  three  levels:  analog,  gate  and 
functional,  or  a  combination  of  them. 

For  the  proposed  system,  a  circuit  may  be  regarded  as  a  set  of  inter¬ 
connecting  blocks.  For  each  block,  simulation  can  be  carried  out  at 
various  levels  of  detail,  from  the  abstract  algorithmic  functional  level  to 
the  detailed  electrical  analog  modeling.  Simultaneous  simulation  of  a 
given  block  at  several  levels  can  also  be  performed. 

The  main  advantage  of  having  the  multi-level  capability  is  that  it  enables 
simulation  of  a  system  to  be  performed  in  a  top-down  hierarchical  manner. 
The  entire  system  can  be  simulated  even  though  parts  of  the  design  are 
still  in  different  stages  of  development.  Furthermore,  this  technique 
may  reduce  the  analysis  of  hybrid  (mixed  analog  and  digital)  systems. 
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Simulation  systems  using  this  approach  are  currently  under  development 
by  several  companies,  most  notably  by  Texas  Instruments.  An  earlier 
system  that  had  been  developed  with  a  somewhat  similar  intention  is  the 
LOGIC  IV  simulator  system  by  Westinghouse.  The  system  was  laid  out 
as  a  hierarchical  system  design  tool,  whereby  the  design  engineer  first 
simulates  the  system  at  the  highest  level  as  a  set  of  subroutines  and  then 
later  on  at  lower  levels  in  which  the  various  subroutines  are  replaced  by 
simulated  subassemblies,  such  as  functional  digital  circuits  and  even 
some  analog  circuits.  The  simulator  allows  for  partial  analog  simulation 
because  the  circuit  is  modeled  by  using  a  FORTRAN  subroutine,  and  the 
signal  values  can  be  stored  as  a  36-bit  word  rather  than  as  a  2-bit  number. 

The  mixed-level  simulator  under  development  at  Texas  Instruments, 
called  INTSIM,  can  provide  several  simulation  levels: 

•  Behavioral  (simulation  values  0,  1 ) 

•  Logic  (multiple  simulation  values) 

•  Electrical  (a  modified  version  of  SPICE  2) 

A  hierarchical  Hardware  Description  Language  (HDL)  is  used  to  drive  the 
simulator  and  to  support  descriptions  of  the  design  performance  require¬ 
ments  and  design  description  data  at  every  level,  from  behavioral  and  block 
functional  to  logic  and  electrical.  The  HDL  source  statements  are  then 
compiled  into  a  hierarchical  design  data  base.  The  Circuit  Selection 
Language  (CSL)  is  used  to  reverse  compile  and  generate  the  HDL  source 
from  the  design  data  base  because  it  may  contain  the  design  at  several 
levels  of  detail.  Using  CSL,  the  blocks  to  be  simulated  are  selected  from 
the  design  data  base  and  a  translation  into  simulation  data  structures  is 
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performed.  The  Simulation  Control  Language  (SIMCL)  is  used  to  specify 
the  external  environment,  input  stimuli,  run  options,  the  timing  mode,  and 
the  signals  to  be  displayed.  The  INTSIM  can  then  be  executed  and  the 
results  evaluated.  Changes  can  be  made  to  the  design  (using  HDL),  circuit 
blocks  (using  CSL),  and  external  environment  (using  SIMCL)  and  the 
simulation  repeated. 

INTSIM  is  intended  for  use  in  the  following: 

•  Specification  development 

•  Algorithm  checkout 

•  Verification  of  specific  implementation 

•  Simulation  of  test  patterns 

•  Race/hazard  timing  analysis 

However,  there  are  some  difficult  issues  that  will  have  to  be  resolved 
before  the  mixed-level  simulation  technique  can  be  fully  operational.  At 
present,  a  variety  of  programming  languages  are  needed  to  use  the 
system.  Hardware  Description  Language  (HDL)  is  used  to  describe  the 
design.  Circuit  Selection  Language  (CSL)  is  used  to  select  the  parts  for 
simulation,  and  Simulation  Control  Language  (SIMCL)  is  used  to  specify 
the  external  stimulus  for  simulation.  For  the  system  to  be  user-oriented, 
it  will  require  a  great  deal  of  developmental  effort  to  arrive  at  a 
syntactically  consistent  form  common  to  all  three  languages.  A  more 
detailed  description  of  INTSIM  can  be  found  in  the  references. 


Additional  Information  From  Literature 


An  extensive  search  was  made  of  the  literature  for  simulation  techniques 
that  may  be  applicable  to  UUT  simulation  for  V&V.  The  results  are 
summarized  in  Table  21. 

The  purpose  of  this  table  is  to  help  the  reader  find  information  on  a 
specific  simulation  method.  Entries  in  the  table  can  help  one  determine 
if  the  referenced  article  contains  simulation  information  of  interest.  The 
table  is  organized  as  follows: 

•  The  principal  author's  name 

•  Title 

•  Method  of  simulation 

There  are  five  basic  types  of  simulations  referred  to  in  the  articles.  They 
were  assigned  numbers  1  through  5  corresponding  to  the  following  list: 

1.  Analog  circuit  level  simulation 

2.  Digital  gate  level  simulation 

3.  Functional  simulation 

4.  Interface  level  simulation 

5.  Multilevel  simulation 

Many  articles  contained  information  on  more  than  one  type  of  simulation 
and  therefore  have  several  numbers  given.  The  numbers  are  not  listed 
in  any  significant  order. 
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TABLE  21.  REFERENCE  LITERATURE  SUMMARY 
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Dia  gnosis 


TABI.K  21.  REFERENCE  LITERATURE  SUMMARY  (continued) 


Providing  Testability  of 
Microprocessor  Boards 


TABLE  21.  REFERENCE  LITERATURE  SUMMARY  (concluded) 


•  Context 

The  context  refers  to  what  purpose  the  simulation  serves.  The 
simulation  is  often  used  for  a  specific  purpose  like  automatic  test 
program  generation  (ATPG).  If  no  particular  use  is  given  for  the 
simulation,  the  context  column  is  left  blank.  This  usually  indicates 
that  the  entire  article  was  dedicated  to  simulation. 

•  Comment 

The  comment  describes  the  method  of  simulation  referred  to  in 
the  article.  The  objective  is  to  give  a  summary  view  of  the 
simulation  technique  used. 

APPLICABLE  TECHNOLOGIES 

The  purpose  of  this  section  is  to  discuss  technology  constraints  to  be  used 
for  the  implementation  of  UUT  simulators  for  V&V.  The  current  rapid 
advancement  of  digital  technology  dictates  that  specific  periods  be  targeted 
for  the  simulator  implementation.  Otherwise,  if  a  technology  is  chosen 
that  is  too  near- in,  not  enough  performance  may  be  available;  if  a 
technology  that  is  too  far  out  is  chosen,  projected  performance  may  be 
adequate,  but  too  many  additional  risk  factors  would  be  brought  into  the 
development  of  the  simulator. 

There  are  basically  two  time  frames  of  interest  for  a  UUT  simulator  in 
support  of  the  MATE  program.  A  UUT  simulator  demonstration  system 
to  be  available  in  the  1983  time  frame  must  use  technology  that  is 
available  in  the  1981  time  frame.  A  generic  UUT  simulator  system  in  the 
post-1985  time  frame,  may  be  recommended  as  a  result  of  the  demonstra¬ 
tion  system  to  develop. 


155 


To  help  discuss  the  technology  constraints,  the  following  set  of  categories 
will  be  used: 

•  Computer  Hardware 

•  Interface  Devices 

•  Communication  Support 

•  System  Architecture 

•  Software 

•  Data  Bases 

Computer  Hardware 


Processors  — For  the  1983  time  frame,  a  wide  range  of  processors  ranging 
from  bit  slice  Arithmetic  Logic  Units  through  8-  and  16-bit  micros  to 
16-  and  32-bit  minis  will  be  capable  of  being  used.  All  these  technologies 
are  in  place  today  to  varying  degrees  of  completeness.  The  16-bit  micros 
should  be  available  for  use  by  the  1981  time  frame.  Augmentation  support 
chips  (such  as  floating  point  processors)  are  also  available  to  support 
building  special  architectures  for  increasing  simulation  speeds.  A  speed 
range  of  0.  5  to  4  million  instructions  per  second  should  be  supportable. 

For  the  post-1985  time  frame,  a  full  range  of  8-,  16-,  and  32-bit 
conventional  processors  as  well  as  special  purpose  processors  such  as 
floating  point  units  and  signal  processing  and  array  processing  units  can 
be  assumed  available.  Speed  ranges  in  the  1  to  10  million  instructions 
per  second  range  can  be  assumed. 
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Main  Memory— For  the  1983  time  frame,  16K  to  64K  RAM  chips  with 
100  to  300  nsec,  access  can  be  assumed.  Although  the  64K  RAM  chips  are 
not  yet  available,  they  can  be  assumed  ready  for  use  in  1982.  The  main 
implication  in  the  use  of  a  UUT  simulator  is  that  large  main  memories  can 
be  assumed  available  to  the  extent  they  are  needed. 

For  the  post-1985  time  frame,  64K  to  256K  RAM  chips  can  be  assumed 
available  with  further  improved  access  speeds. 

«.»  •  *  ■  •  •  ••  •  •  • 

Mass  Storage— In  the  1983  time  frame,  floppy  disks  with  capacity  ranges 
of  1  00K  bytes  to  2M  bytes  can  be  assumed  available  to  support  storage  of 
UUT  simulator  characteristics.  Cartridge  disks  and  hard  disks  with 
storage  capacities  in  the  range  10M  to  500M  bytes  can  also  be  assumed 
available  if  large  data  bases  are  needed.  However,  storage  requirements 
beyond  the  20M  tc  50M  byte  range  may  imply  that  the  implemented  system 
is  no  longer  portable. 

In  the  post-1985  time  frame,  further  increases  in  capacities  are  expected. 
Furthermore,  magnetic  bubble  mass  storage  devices  in  the  1M  byte  to 
50M  byte  range  can  be  assumed  available  and  cost-effective  if  the  access 
time  of  the  rotating  media  devices  becomes  a  constraining  factor  in  the 
UUT  simulator  performance. 

Peripherals  — A  UUT  simulator  is  not  expected  to  require  unusual 
peripherals.  Standard  tapes  may  be  desirable  to  support  backup  and  to 
capture  trace  information.  Conventional  printers  and  CRTs  are  assumed 
adequate  for  operator  and  user  interfaces. 
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Interfaces  —  The  main  UUT  simulator  interface  devices  of  concern  are 

analog  to  digital  and  digital  to  analog  converters.  If  a  simulator  solution 

requires  interception  and  generation  of  direct  analog  signals,  the  accuracy, 

the  speed,  and  the  drive  /sink  capabilities  of  the  converters  are  important. 

In  the  near  term,  one  can  assume  12-bit  A/D  converters  with  speeds  of 

1  MHz  to  5  MHz  are  cost-effective  to  use  in  a  simulator.  Converters  with 

less  accuracy  can  run  at  higher  speeds.  D/A  converters  can  run  at 

higher  speeds  than  A/Ds,  but  the  nature  of  their  output  is  more  critical 

to  their  performance.  To  make  full  use  of  high  converter  speeds,  very 
■  •  «  • 

high-speed  simulation  processing  would  be  required.  The  specific 
characteristics  of  the  converters  are  less  of  a  concern  because  the 
processing  power  would  probably  become  a  UUT  simulator  system  bottle¬ 
neck  much  earlier  than  the  speed  at  which  the  conversions  are  performed. 

Communication— In  the  1983  time  frame,  conventional  communication  chips 
supporting  RS232,  IEEE-488,  and  other  special  communication  protocols  can 
be  assumed  available  to  support  interconnecting  sub-systems  of  a  UUT 
simulator  if  desired. 

In  the  post-1985  time  frame,  higher  bandwidth  optical  links  should  be 
available.  High-speed  direct  links  to  large  data  bases  (such  as  would 
contain  device  characteristics  and  UUT  design  information)  can  be 
assumed  available. 

System  Architecture— In  the  1983  time  frame,  the  following  ranges  of 
system  architectures  can  be  assumed  available  for  use  in  a  UUT  simulator: 
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•  Single  computer  systems  (simplest  case) 

•  Multi- computers  cooperating  on  simulation  support 

•  Limited  forms  of  distributed  processing 

More  advanced  architectures  such  as  parallel  processors,  pipelined 
processors  and  multi-micro-implemented  simulators  should  not  be 
considered  in  this  time  frame  because  they  would  introduce  additional 
risks  in  demonstrating  and  evaluating  the  UUT  simulator  concepts. 

In  the  post-1985  time  frame,  all  of  the  above  advanced  architectures  can 
also  be  used.  In  addition,  it  can  be  assumed  that  VLSI  implemented 
"Direct  Execution  Machines"  specifically  oriented  to  supporting  UUT 
simulations  can  be  built  into  system  architectures.  These  components 
would  both  improve  the  response  times  of  the  simulators  and  allow  more 
cost-effective  use  of  the  simulation  system. 

Software— Programs  developed  for  a  1983  time  frame  system  would  have 
to  rely  on  software  technology  already  currently  in  place.  Since  the  UUT 
simulator  would  more  than  likely  be  based  on  a  minicomputer,  the 
implication  is  that  the  software  would  have  to  be  FORTRAN -based.  Test 
software  is  assumed  to  be  based  on  ATLAS.  Not  much  change  for 
conventional  support  software  available  today  is  expected  during  the  next 
few  years. 

In  the  post-1985  time  frame,  the  DoD  high  order  language  ADA  should  be 
sufficiently  mature  and  adequately  supported  to  assume  it  could  be  used  in 
further  UUT  simulator  software  development. 
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Data  Bases— In  the  post-1985  time  frame,  it  can  be  assumed  that  large 
data  bases  supporting  UUT  component  models,  UUT  design  data,  and  test 
program  information  will  likely  be  available  and  conveniently  accessible. 
The  UUT  simulator  should  therefore  be  capable  of  being  built  and  driven 
by  information  from  these  data  bases.  The  short  term  implication  is  that 
the  UUT  simulator  design  should  be  modular  and  not  tied  in  to  a  specific 
method  of  simulation. 

SOLUTION  CATEGORIES 

Now  that  we  have  the  UUT  simulator  requirements,  the  applicable  simula¬ 
tion  techniques,  and  the  assumed  technology  constraints,  we  can  proceed 
to  postulate  what  the  UUT  simulator  system  will  look  like.  Because  there 
are  many  possible  solutions,  some  method  of  structuring  them  must  be 
used  in  order  to  talk  about  them  intelligently.  The  strategy  we  used  was 
to  look  at  the  solutions  in  terms  of  orthogonal  attributes. 

The  first  dimension  of  this  attribute  space  was  assumed  to  be  the  level 
of  simulation  used  in  the  UUT  simulator: 

1.  Analog  Circuit  Level 

2.  Digital  Gate  Level 

3.  Functional  Level 

4.  Interface  Level 


5.  Multi-Level 


The  first  4  levels  represent  decreasing  levels  of  fidelity  as  one  proceeds 
from  the  analog  to  the  interface  level.  The  last  case  represents  a 
combination  of  simulation  methods  used  concurrently  (i.  e. .  different  parts 
of  the  TUT  are  simulated  using  different  levels). 

Figure  4  illustrates  the  relationships  of  these  levels  of  simulation  in  a 
digital  or  hybrid  UUT.  The  analog  circuit  level  of  simulation  is  needed 
when  the  simulated  faults  are  involved  with  circuit  components  or  when 

the  measurements  involve  time  constants  which  are  close  to  the  time  .  . . 

constants  inherent  in  the  circuit.  This  level  would  be  needed  for  measuring 
rise  times,  overshoots,  and  effects  of  switching  spikes  in  some  forms  of 
digital  circuits. 


ANALOG  CIRCUIT  LEVEL 

(CIRCUIT  TIME  CONSTANT  CONTROLLED) 


Figure  4.  Levels  of  Simulation 
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The  digital  gate  level  of  simulation  is  needed  if  the  UUT  contains  a  lot  of 
digital  logic  or  if  internal  digital  faults  are  to  be  simulated  in  larger 
system  functions.  The  simulation  proceeds  at  about  the  gate  delay  level  of 
time  increments,  considerably  faster  than  the  analog  level  simulation. 

Some  forms  of  gate  level  simulation  have  enough  fidelity  to  allow  simulation 
timing  analyses,  such  as  in  looking  for  race  conditions;  others  do  not. 

If  multi-level  simulation  is  supported  in  the  UUT  simulator,  situations 
requiring  greater  detail  than  supportable  at  the  gate  level  can  be 
simulated  at  the  circuit  level. 

The  functional  level  of  simulation  involves  dynamic  input/output  characteris¬ 
tics  of  subcomponents.  In  a  digital  system,  the  functional  level  would 
correspond  to  calculating  only  the  outputs  at  the  chip  pins  for  every  clock 
period.  Signal  details  between  these  times  and  details  of  internal 
behavior  are  suppressed.  Computation  time  is  reduced  but  each  assumed 
different  internal  fault  would  require  another  function  implementation  to 
simulate  its  effects. 

The  interface  level  of  simulation  is  a  special  case  of  the  functional  level. 
Only  static,  typically  table-driven,  input/output  functional  relationships 
are  assumed.  New  sets  of  inputs  as  well  as  different  faults  imply  additional 
sets  of  entries  in  the  input/output  relationship  table. 

A  second  dimension  of  the  UUT  simulator  attribute  space  is  the  method 
of  interfacing  the  UUT  simulator  to  the  automatic  test  equipment  system 
which  is  executing  the  test  software  to  be  verified.  The  nature  of  the 
interface  has  a  strong  impact  on  the  simulator  constraints,  cost,  and  the 
extent  to  which  the  simulator  can  displace  the  need  for  the  real  UUT. 
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Figure  5  shows  the  three  ranges  of  interfaces  which  we  feel  cover  UUT 
simulator  solutions  of  interest.  The  first,  the  analog  signal  interface, 
provides  the  most  independent  form  of  interface.  It  assumes  that  the 
actual  interface  adapter  for  the  UL'T  will  be  used,  and  the  real  UUT  is 
mimicked  at  the  interface  pin  level.  The  solution  requires  A/D  converters, 
level  shifters,  isolators,  etc.  as  may  be  required  to  measure  input 
signals,  generate  realistic  responses,  and  isolate  the  simulator  system 
from  the  test  equipment. 

Figure  6  gives  a  pictorial  representation  of  what  this  simulator  may  look 
like  and  how  it  would  be  connected  to  the  ATE  system.  Depending  on  the 
range  of  UUTs  simulated,  the  simulator  may  have  to  sink  much  power  or, 
conversely,  generate  it  to  drive  the  loads  switched  in  on  the  test  equipment 
side.  To  be  anywhere  near  generic,  this  method  of  interfacing  may  require 
as  much  test  equipment  as  the  ATE  system  itself. 

The  realism  of  this  form  of  interface  is  offset  by  the  following  negative 
aspects  of  the  interface: 

•  Only  low  bandwidth  UUTs  may  be  capable  of  being  simulated. 

•  High  throughput  simulation  processing  is  needed  to  provide 
real-time  responses.  The  instruments  at  the  ATE  side  of  the 
interface  may  require  full  speed  responses  to  make  the  proper 
measurements. 

•  Signal  speed  is  limited  by  the  bandwidth  of  the  A/D  and  D/A 
converters.  Usually  the  simulation  processing  would  saturate 
much  earlier  than  the  converters,  but  some  forms  of  simulations 
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Figure  6.  Drawing  of  UUT  Simulator  (Analog  Level) 


such  as  the  interface  level  may  produce  waveforms  fast 
enough  to  reach  the  speed  limits  of  the  converters. 

•  Extraneous  noise  and  grounding  problems  may  be  introduced, 
which  may  detract  from  the  test  program  verification  activity. 
These  problems  may  produce  undesirable  responses  for  test 
programs  and  the  developers  or  validators  may  be  forced  into 
chasing  problems  which  do  not  even  exist  in  the  real  UUT. 

•  If  the  test  program  contains  bugs  which  are  potentially  damaging 
to  the  real  UTJT,  these  same  bugs  are  also  potentially  damaging 
to  the  equipment  supporting  the  UUT  simulator  interface. 

A  side  benefit  of  the  analog  signal  interface  is  that  it  does  allow  the  testing 
of  the  interface  adapter  before  it  is  connected  to  the  real  UUT. 

The  second  interface  alternative  is  the  instrument  bus  interface.  In  this 
case  we  assume  that  the  ATE  system  is  modular  and  has  a  well-defined 
bus  which  controls  the  test  equipment  instruments.  The  strategy  is  to 
connect  the  simulator  system  to  the  ATE  CPU  at  this  bus.  In  addition  to  the 
UUT  simulator,  this  interface  also  requires  the  following: 

•  A  simulator  for  the  test  equipment.  This  simulator  would 
interpret  commands  from  the  instrument  bus,  generate  the 
signals  as  commanded,  and  measure  the  simulated  responses  as 
requested.  A  functional  simulator  has  to  be  developed  for  each 
piece  of  test  equipment. 

•  A  simulator  for  the  interface  adapter.  This  simulator  would 
have  to  be  developed  to  properly  represent  the  routing  and  any 
potentially  active  functions  in  the  real  interface  adapter.  Only 
a  functional  level  simulator  would  be  needed. 
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Figure  7  shows  a  pictorial  representation  of  the  instrument  bus  interfaced 
solution.  Note  that  neither  the  test  equipment  bay  of  the  ATE  or  the 
special  analog  instruments  of  the  simulator  for  the  analog  interface  case 
are  needed.  The  bus  interface  solution  basically  supports  two  computers 
talking  to  one  another,  with  information  represented  at  all  stages  in 
digital  form.  The  criticality  of  the  timing  is  also  removed  as  compared 
to  the  analog  interface  case.  The  simulation  can  proceed  at  a  slower  than 
real-time  rate.  Timing  constraints  would  be  encountered  only  if  the  ATE 
test  programs  contain  time-outs  which  cannot  be  deactivated  or  slowed  down. 

A  third  interface  alternative  is  the  software-only  solution.  In  this  case 
both  the  interpreter  for  the  test  program  and  the  UUT  simulator  are 
executed  in  the  same  computer.  There  are  now  a  number  of  additional 
options  in  handling  the  interface.  The  test  equipment  and  the  interface 
adapter  could  be  simulated,  as  in  the  bus  interface  case,  but  at  a  less 
precise  level.  In  addition,  the  semantics  of  the  test  program  statements 
could  be  executed  directly  at  the  UUT  pin  level  without  regard  to  the 
specific  test  equipment  used  to  generate  the  stimulus  or  the  nature  of  the 
interface  adapter.  A  consequence,  though,  is  that  no  verification  of  the 
interface  adapter  is  performed. 

SOLUTION  COMPONENTS 

To  further  differentiate  alternative  UUT  simulator  solutions,  we  can  look  in 
more  detail  at  specific  components  that  would  make  up  a  UUT  simulator. 

We  will  then  find  that  some  components  are  common  among  classes  of 
solutions  and  some  are  unique  to  specific  classes.  By  examining  these 
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components  in  more  detail  we  can  get  a  grasp  on  both  development  and 
usage  costs  as  needed  to  perform  trade-offs  between  solutions. 

Figure  8  shows  an  overview  of  a  signal  flow  model  of  a  UUT  simulator. 

By  following  signals  through  these  components  we  can  get  a  better  feeling 
of  the  interaction  of  the  components. 

The  input  from  the  ATE  system  first  passes  through  the  set  of  functions 
labeled  "input  Signal  Recognizer,  "  "Test  Equipment  Stimuli  Generator,"  and 
and  "Interface  Adapter  Input  Simulator".  The  flow  as  shown  corresponds 
to  the  bus  interface  set  of  solutions.  For  the  analog  interface  solution 
only  the  signal  recognizer  (consisting  of  both  hardware  and  software) 
would  be  required. 

The  signal  then  flows  into  the  UUT  simulation  control  function  which  has 
the  job  of  passing  it  to  the  proper  level  of  simulation.  The  figure  shows 
the  case  in  which  all  four  levels  of  simulation  are  present.  If  only  one 
level  is  supported,  the  routing  is  simplified. 

The  simulation  of  the  UUT  then  performs  transformations  which  ultimately 
result  in  simulated  outputs.  These  outputs  are  then  routed  via  the  interface 
adapter  output  simulator,  the  test  equipment  measurement  model,  and  the 
output  signal  generator  back  to  the  test  equipment.  This  flow  again 
corresponds  to  the  bus  interface  solution.  For  the  analog  interface 
solution  only  the  output  signal  generator  would  be  required. 
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The  two  functions  shown  in  the  diagram  that  are  not  yet  discussed  are 
the  Operator  Input/Output  and  the  Model/ Activity  Control.  The  operator 
input/output  is  required  if  the  UUT  has  outputs  that  must  be  manually 
observed  or  inputs  such  as  function  switches.  The  model/activity 
control  is  used  for  controlling  the  simulation  and  for  selecting  model 
parameters  and  inserting  faults. 

Figure  9  shows  a  simpler  signal  flow  diagram  but  emphasizing  the 
software-only  form  of  interfacing.  In  this  case  the  ATLAS  interpreter 
is  needed  in  place  of  the  test  equipment  and  interface  adapter  simulators. 

A  main  point  to  be  observed  is  that  the  right  side  of  the  diagram  containing 
the  simulations  as  well  as  the  operator  interaction  is  the  same  as  in  the 
previous  diagram. 

Figure  10  shows  a  hierarchical  perspective  of  the  components  of  a  UUT 
simulator.  The  purpose  of  looking  at  these  components  in  more  detail  is 
to  allow  us  to  assess  their  impacts  on  the  various  possible  forms  of 
UUT  simulators  with  regard  to  development  costs  and  usage  costs. 
Comments  on  cost  implications  are  given  for  most  components. 

Simulation  Sequencing  Control 


This  is  the  main  controlling  activity  in  the  UUT  simulator.  It  must  ensure 
that  the  resources  in  the  computer  supporting  the  simulation  are  shared 
among  the  subcomponents  of  the  simulator.  It  must  also  control  the  timely 
activation  of  components  to  ensure  timing  constraints  are  not  violated. 
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Figure  9.  Software-Only  Form  of  UUT  Simulator 


Development  costs  are  similar  for  all  simulation  levels.  The  analog 
interface  solutions  require  extra  hardware  support  to  meet  stringent 
real-time  constraints.  The  bus  interface  solutions  require  an  intermediate 
level  of  complexity  for  this  function,  and  the  software-only  interface 
solutions  require  only  minimal  support. 

Usage  costs  are  similar  for  all  simulation  levels.  The  analog  interface 
solutions  imply  higher  usage  costs  because  of  the  more  complex  and 
higher  real-time  demands  for  controlling  the  sequencing. 

UUT  Simulation  Control 

This  function  controls  the  activation  and  coordination  of  the  simulation 
models.  It  must  support  matching  time  increments  when  more  than  one 
level  of  simulation  is  used.  It  must  also  support  interfacing  signal 
levels  between  the  different  simulations. 

For  the  simulation  levels  involving  only  one  type  of  simulation,  the  control 
functions  are  relatively  simple  and  the  development  costs  should  be  low. 
For  the  multilevel  simulation  case,  extensive  control  and  interfacing  are 
needed  to  realize  the  benefits  of  using  all  the  simulation  levels.  The 
development  costs  are  independent  of  the  interface  method. 

Only  the  multilevel  simulation  case  would  require  a  significant  usage  cost. 
These  costs  should  be  more  than  compensated  for  through  the  gains  in 
efficiency  in  performing  the  total  simulation.  The  usage  costs  are 
independent  of  the  interface  method. 
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Analog  Circuit  Simulation 


This  component  performs  the  analog  simulations  for  all  the  parts  of  the 
UUT  to  be  simulated  at  this  level.  It  is  assumed  that  some  existing 
system  for  analog  circuit  simulation  is  either  used  directly  or  adapted 
to  specific  requirements  for  performing  UUT  simulations.  The  simulation 
is  ideally  compatible  with  forms  of  simulation  used  for  the  UUT  during 
other  phases  of  its  development.  For  example,  if  analog  simulation  is 
used  during  design,  it  is  desirable  to  adapt  this  simulation  for  use  as 
a  UUT  simulator  oriented  to  the  support  of  Verification  and  Validation. 

The  choice  of  which  analog  simulator  to  be  used  must  be  based  on  more 
specific  system  requirements  and  development  contexts. 

For  the  simulator  solutions  involving  either  the  analog  simulation  level 
only  or  the  multilevel  simulation,  the  development  costs  are  expected  to 
be  high.  It  can  take  as  much  as  a  few  man-years  to  adapt  an  existing 
analog  simulation  system  to  a  new  computer  environment. 

Adapting  an  analog  simulator  for  use  as  a  UUT  simulator  (particularly  one 
implemented  on  a  minicomputer)  would  require  both  the  adaptation  to  the 
new  environment  and  the  addition  of  support  to  facilitate  topology  and 
circuit  component  changes  to  simulate  the  insertion  of  faults.  For  the 
UUT  simulator  solutions  not  involving  analog  simulation,  there  are  no 
associated  development  costs.  The  costs  are  also  independent  of 
interface  methods.  But  if  the  analog  interface  is  to  be  supported  and 
high  simulation  speeds  are  required,  much  additional  development  is 
required  to  re-implement  the  analog  simulation  on  higher  speed  processors 
(such  as  special  microprogrammed  processors  or  array  processors). 
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For  the  simulator  solutions  involving  only  analog  simulations,  the  usage 
costs  are  expected  to  very  high.  Either  long  periods  of  a  conventional 
computer  are  required  or  special  (and  expensive)  equipment  must  be 
procured  and  amortized  during  the  use  of  the  simulator.  For  the  multi¬ 
level  simulation  solutions,  costs  are  expected  to  be  moderate  because 
only  parts  of  a  TJUT  would  have  to  be  simulated  at  the  analog  level.  The 
usage  costs  are  again  independent  of  the  interface  method  unless  the  analog 
interface  solutions  require  high-speed  processing.  In  this  case  the 
simulation  costs  are  expected  to  be  very  high. 

Gate- Level  Simulation 


This  component  performs  the  gate  level  simulations  for  all  parts  of  the  UUT 
to  be  simulated  at  this  level.  It  is  again  assumed  that  some  existing  gate 
level  simulator  (typically  from  a  digital  test  program  generator)  is  available 
for  adaptation  to  the  UUT  simulator  environment.  Fewer  internal  modifica¬ 
tions  are  likely  needed  because  the  fault  insertion  mechanisms  are  simpler 
and  may  already  be  built  into  the  simulator. 

In  those  simulator  solutions  where  gate  level  simulation  is  needed,  the 
development  costs  are  assumed  to  be  moderate.  The  assumption  is  that 
some  existing  simulator  can  be  modified  and  tailored  for  the  UUT 
sinulator  environment.  The  development  cost  is  independent  of  the  inter¬ 
face  method  used  as  long  as  no  extraordinary  speedups  are  required  to 
meet  throughput  rates  higher  than  the  original  simulator  was  designed  for. 
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I'sage  costs  would  tend  to  be  moderate  to  high,  depending  on  the  amount 
of  gate  level  simulation  needed  in  specific  simulations.  The  costs  would 
probably  range  between  the  cost  of  doing  more  detailed  analog  simulations 
and  the  cost  of  doing  functional  level  simulations. 

Functional  Simulations 


The  general  support  provided  at  this  level  of  simulation  is  assumed  to  be 
minimal.  At  most,  some  primiti  es  and  low  level  functions  supporting 
functional  simulation  would  have  to  be  provided  a  priori  to  the  actual 
simulations.  Therefore  the  one-time  development  costs  are  low.  However, 
for  each  ITT  to  be  simulated  at  this  level,  an  extensive  development  may 
have  to  be  undertaken  to  build  up  both  the  properly  operating  functional 
simulation  and  functions  for  simulating  fault  models. 

Development  costs  are  expected  to  be  low. 

The  cost  of  running  the  simulated  functions  will  be  relatively  low  compared 
to  gate  level  or  analog  simulation.  However,  a  one-time  development 
effort  must  be  planned  for  each  TT'T.  This  is  expected  to  be  much  more 
costlv  than  entering  the  models  and  topologies  of  an  analog  or  digital 
simulation. 
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Interface  Level  Simulation 


It  is  assumed  that  this  form  of  simulation  would  require  some  form  of 
preprocessor'  or  simple  special  purpose  language  to  support  entering 
input /output  relationships  to  be  simulated.  Simple  dynamic  cases,  such 
as  sequences  of  signals,  should  also  be  supportable  for  the  interface  level 
of  simulation  to  be  effective. 

A  low  to  moderate  development  cost  would  be  incurred  (as  compared  to 
an  analog  or  digital  gate  simulation).  For  ITT  simulator  solutions 
containing  only  nterface  level  simulation,  the  development  should  be  more 
complete  than  if  other  forms  of  simulations  are  also  supported.  In  the 
latter  case,  more  difficult  simulations  can  be  done  in  more  detailed 
si  muiations. 

I  sage  costs  are  expected  to  be  low  for  all  cases. 

Output  Signal  Oenerator 


The  function  of  this  component  is  to  match  the  output  of  the  simulator  to 
the  input  of  the  \TF  system  at  the  proper  interface  level.  This  component 
would  be  completely  different  for  the  different  methods  of  interfacing.  For 
the  analog  interface  methods,  both  hardware  and  software  along  with  real- 
time  constraints  are  implied.  Consequently,  both  the  development  costs 
and  tin  usage  costs  are  high.  The  software-only  sits  of  solutions  would 
be  least  costly  because  there  are  no  real-time  constraints,  and  the  formats 
can  be  designed  to  be  compatible.  The  instrument  bus  interface  solutions 
require  an  intermediate  level  of  complexity.  The  ITT  simulator  sirnals 
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would  already  have  been  accepted  and  converted  by  the  interface  adapter 
and  test  equipment  simulators.  Therefore,  the  output  signal  generator  has 
to  output  the  response  in  the  proper  protocol  structure  required  by  the 
instrument  bus. 

The  development  costs  are  independent  of  the  levels  of  simulation.  As 
mentioned  above,  the  analog  interface  would  require  costly  hardware  and 
software  developments.  The  software-only  solutions  would  imply  the 
least  cost. 

The  analog  interface  solutions  would  imply  relatively  high  usage  costs, 
considering  higher  maintenance  and  amortization  of  the  additional  hardware. 

Input  Signal  Recognizer/Converter 

The  function  of  this  component  is  to  convert  the  signals  from  the  ATE 
system  into  internal  formats  appropriate  for  the  simulation  inputs.  The 
specific  functionality  of  this  component  is  strongly  dependent  on  both 
the  simulation  technique  and  the  interface  method: 

•  For  the  analog  interface  method,  the  component  consists  of 
both  hardware  and  software,  implying  high  costs  for  both 
development  and  usage. 

•  For  the  bus  interface  method,  this  component  consists  of 
simple  digital  hardware  and  software  to  handle  the  instrument 

bus  protocol.  Both  development  and  operation  costs  would  be  less. 
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•  The  software-only  solution  would  be  least  complex  and  least 
costly. 

•  Both  development  and  operation  costs  would  be  similar  for  all 
levels  of  simulation  except  the  interface  level  simulation  method. 
In  this  case  the  input  recognizer  would  be  required  to  recognize 
sets  of  input  signals  in  order  to  generate  the  proper  outputs.  The 
implications  are  that  both  the  development  and  operations  costs 
for  the  input  signal  recognizer  would  be  higher  than  for  the  other 
simulation  techniques. 


Operator  Interface  Simulator 


The  function  of  this  component  is  to  display  any  UUT  outputs  which  the 
operator  is  expected  to  observe  and  to  accept  operator  inputs  (such  as 
via  soft  keys)  when  requested  by  the  test  program.  It  is  assumed  adequate 
to  represent  the  UUT  outputs  via  alphanumerics  on  the  UUT  simulator 
console. 


Development  costs  are  similar  for  all  simulator  solutions. 


Usage  costs  are  insignificant. 
Interface  Adapter  Simulator 


This  component  is  needed  for  the  bus  interface  set  of  solutions.  It  must 
be  developed  specifically  for  each  UUT,  and  it  should  be  based  on  the  design 
or  schematics  for  the  real  Interface  Adapter  used. 
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Initial  UUT  simulator  development  costs  are  low 


A  one-time  development  cost  is  incurred  for  each  UUT,  but  once  developed, 
the  usage  costs  should  be  insignificant. 

ATE  Equipment  Simulator 

This  component  is  needed  in  the  solutions  using  a  bus  interface.  The 
simulator  is  driven  by  commands  received  by  the  input  recognizer  and 
simulates  stimuli  generation  and  performance  of  measurements.  The 
initial  UUT  simulator  development  would  require  the  development  of 
simulators  for  a  basic  set  of  test  equipment  functions  and  instruments. 
Additional  instrument  simulations  must  be  developed  as  new  instruments 
are  used  for  new  UUTs.  However,  functional  simulations  should  be 
adequate. 

Initial  UUT  simulator  development  costs  depend  on  the  complement  of 
instruments  assumed  supported  by  the  ATE. 

After  a  sufficient  number  of  UUT  simulators  have  been  developed  using 
the  same  system,  new  instrument  simulations  should  have  to  be  added 
only  infrequently.  The  run-time  usage  costs  should  be  insignificant. 

Model/ Activity  Control 


The  function  of  this  component  is  to  accept  user  inputs  to  control  the 
progress  of  the  UUT  simulation.  It  is  shown  as  the  top  level  control  for 
the  following  three  lower  level  functions. 
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Console  Control- -This  component  processes  console  input  commands  and 
supports  the  output  of  information  to  the  console.  Examples  of  lower 
level  functions  supported  are  command  decoding  and  format  conversions. 
User  support  functions  needed  to  debug  UUT  simulators  during  their 
development  (such  as  traces,  dumps,  and  breakpoints)  must  also  be 
provided. 

Development  costs  are  independent  of  the  interface  method  or  level  of 
simulation. 

Usage  costs  are  negligible. 

Model  Switching  Control- -This  component  is  needed  for  only  the  multi-level 
simulation  solutions.  Its  function  is  to  control  the  switching  in  and  out  of 
the  different  simulation  models  in  order  to  support  faster  total  simulation 
of  time.  Part  of  the  function  of  this  component  also  consists  of  supporting 
the  interfacing  of  different  UUT  component  models  when  transitions  in 
levels  are  made. 

Moderate  development  costs  can  be  assumed  for  the  multi-level  simulation 
solutions. 

Usage  costs  are  negligible. 

Fault  Isolation  Control- -This  component  supports  the  control  over  inserting 
faults  or  changing  models  to  produce  the  symptoms  of  a  fault.  Semi¬ 
automatic  control  is  assumed  to  help  expedite  cycling  through  the  process 
of  starting  a  test,  inserting  a  fault,  watching  the  ATE  system  react,  and 
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resetting  the  UUT  simulator  to  prepare  for  insertion  of  the  next  fault. 

This  is  one  of  the  key  support  functions  in  the  UUT  simulator  system  for 
aiding  the  independent  V&V  process. 

Costs  are  dependent  on  specific  simulation  techniques. 

Usage  costs  are  dependent  on  simulation  techniques  and  are  comparatively 
low. 

Common  Support  Modules 


This  component  consists  of  a  collection  of  low  level  functions  needed  by 
many  of  the  higher  level  UUT  simulator  environment  modules.  A  reason 
to  separate  them  is  to  maximize  commonality  and  therefore  reduce  both 
development  costs  and  later  maintenance  costs. 

Development  costs  are  low  to  moderate. 

Usage  costs  are  low. 

Model  Data  Base  Support 

This  component  supports  information  structures  needed  for  data-base- 
driven  simulations  such  as  an  analog  simulation  and  a  gate  level  simulation. 
The  ability  to  separate  this  module  out  as  a  separate  component  depends  on 
the  specific  simulation  techniques  chosen  and  how  the  simulators  were 
developed. 
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Other  Support  Components 


In  addition  to  the  UUT  simulator  components  required  during  the  actual 
running  of  a  simulation,  another  set  of  support  components  is  needed  to 
prepare  for  the  execution  of  the  simulation.  These  components  are  par¬ 
titioned  and  illustrated  in  Figure  11. 

The  main  cost  associated  with  these  components  is  development  costs. 

The  usage  costs  can  be  considered  negligible.  A  brief  description  of  each 
of  the  components  is  given  below: 

Interaction  Model  Builder—  This  is  a  postulated  component  that  takes  a 
source  description  of  the  UUT  to  be  simulated  and  builds  a  data  base 
describing  how  the  UUT  components  interact.  It  would  perform  consistency 
checks  as  the  data  base  is  built.  The  nature  of  this  component  is  again 
very  dependent  on  the  forms  of  simulations  used  in  the  UUT  simulator 
systems. 

Interface  Model  Preprocessor— It  is  assumed  that  the  interface  level  of 
simulation  would  have  to  be  supported  by  only  a  simple  preprocessor  or 
at  most  a  simple,  special  purpose  language  translator.  The  function  of 
this  preprocessor  is  to  accept  the  input/output  descriptions  comprising 
the  UUT  component  models  and  to  build  the  internal  table  in  a  form 
compatible  with  processing  software. 
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Figure  11.  UUT  Simulator  Support  Functionality 


Functional  Model  Integrator— This  component  supports  the  coalescing  or 
binding  of  independently  developed  modules.  The  most  rudimentary 
version  of  this  model  is  simply  the  link  editor  of  the  host  computer  support 
system.  In  this  case  the  functional  modules  are  assumed  programmed  in 
a  high  order  language  supported  by  the  host. 

Gate  Level  Logic  Translator— If  this  form  of  simulation  is  used  in  the  UUT 
simulator,  this  component  performs  the  translation  of  register  transfer 
or  gate  level  circuit  descriptions  into  a  form  compatible  with  the  internal 
simulation  data  base.  This  translator  would  normally  be  provided  by 
the  gate  level  simulation  system  incorporated  into  the  UUT  simulator 
system. 

Analog  Model  Translator— This  component  would  be  typically  considered 
as  part  of  the  analog  simulation  system.  It  accepts  descriptions  of 
analog  circuit  models,  performs  checks,  and  builds  the  data  base  for 
the  analog  simulated  sections  of  the  UUT. 

Interface  Adapter  Model  Builder— If  the  simulation  system  requires 
models  for  the  interface  adapter,  some  form  of  preprocessor  support  is 
needed.  Because  the  models  must  be  functional  at  most,  the  preprocessing 
should  be  simple. 

UUT  Simulator  Data  Base  Control— This  component  provides  common 
data-base  construction  support  and  access  to  the  data  base,  and  performs 
checks  as  well  as  data  compression.  Ideally  the  data  base  control  is 
common  for  all  the  simulation  techniques.  The  ability  to  accomplish  this 
and  the  structure  of  this  component  depend  largely  on  the  specific  simulation 
approaches  and  implementations  chosen. 
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ATE  Equipment  Model  Builder— If  the  simulation  system  requires  models 
for  the  ATE  equipment,  some  form  of  support  is  needed  to  develop  the 
internal  data  base  models.  The  models  need  again  to  be  at  only  the 
functional  level,  and  therefore  the  support  required  for  this  function  is 
assumed  to  be  minimal. 

Support  Functionality  Control— This  component  (not  shown  on  Figure  11) 
includes  high  level  control  of  the  UUT  simulator  support  functionality. 

It  includes  functions  to  allow  user  interfacing,  to  perform  checks,  and  to 
control  activation  of  the  specific  lower  level  functions. 

SOLUTION  SELECTION 

The  set  of  UUT  simulator  solutions  covered  by  the  component  descriptions 
discussed  above  covers  a  very  wide  range.  To  zero  in  on  the  most  cost- 
effective  solution  an  appropriately  scoped  trade-off  was  performed.  The 
initial  direction  of  this  study  was  to  determine  the  relative  benefits  and 
costs  in  terms  of  a  solution  matrix  as  shown  in  Figure  12. 

The  cost  factors  which  were  considered  included  the  following: 

•  UUT  Simulator  System  Development  Cost:  A  one-time  development 
cost  which  is  proportional  to  the  complexity  of  the  components 
required  for  the  UUT  simulator.  Comments  about  the  relative 
complexity  of  the  UUT  simulator  components  were  given  above 
in  the  discussions  of  the  components. 
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Figure  12.  Initial  Trade-off  Matrix  Format 

•  Incremental  UUT  Simulator  Usage  Costs:  The  assumption  in 
these  costs  is  that  the  UUT  simulator  system  has  been  developed 
and  is  stable.  The  usage  costs  now  include  the  procurement 
and  amoritization  of  additional  copies  of  the  simulator  system, 
developing  specific  UUT  simulators,  and  usage  costs  while  the 
simulator  is  used  during  V&V. 

To  arrive  at  estimates  of  the  UUT  simulator  component  costs,  the 
following  development  and  deployment  scenario  was  assumed: 

•  A  generic  UUT  simulator  system  environment  is  initially 
developed  to  conveniently  support  the  later  addition  of  specific 
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UUT  simulators.  This  system  can  be  considered  analogous  to 
a  generic  A-TE  system  that  is  developed  with  general  support 
capability  so  that  specific  instruments  and  test  program  sets 
can  later  be  added. 

•  The  deployment  of  a  specific  TJUT  simulator  system  requires 
that  a  new  copy  of  the  support  hardware  be  procured  and  installed 
at  the  place  in  which  it  is  to  be  used.  It  is  assumed  that  the  cost 
of  this  additional  copy  of  the  system  must  be  amoritized  over 

the  life  of  the  system. 

•  When  a  specific  simulator  for  a  UUT  is  to  be  added,  some 
one-time  development  costs  are  incurred  for  that  UUT.  depending 
on  the  method  of  interfacing  and  the  level  of  simulation. 

•  During  the  use  of  the  UUT  simulator  in  performing  V&V,  both 
the  UUT  simulator  equipment  and  the  test  equipment  which  must 
be  tied  up  while  performing  the  task  must  be  taken  into  account 
in  determining  the  incremental  usage  costs. 

While  the  cost  factors  were  being  developed  for  the  entries  in  the  trade-off 
matrix,  it  was  found  that  most  factors  were  dependent  on  either  the 
simulation  level  or  the  interface  method.  Consequently,  it  was  decided  to 
reduce  the  number  of  entries  in  the  trade-off  by  separating  the  simulation 
levels  and  the  interface  methods. 

Table  22  contains  the  development  cost  estimates  for  the  UUT  simulator 
components.  The  numbers  entered  represent  estimates  of  the  number  of 
statements  in  the  source  code  required  to  implement  the  component  or 
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equivalent  source  code  statements  if  hardware  is  involved.  These 
numbers  represent  a  rough  order  of  magnitude  estimates  based  on 
the  authors'  perception  of  the  component  and  on  experience  in  working 
with  or  developing  similar  functionality  components.  The  objective  of 
this  exercise  was  to  get  a  first  cut  at  providing  inputs  for  the  trade-off 
study.  These  figures  of  the  number  of  statements  were  converted  to 
dollars  by  assuming  a  $50  per  statement  cost  of  software  development. 

Table  23  contains  a  break-out  of  the  cost  components  factored  into  the 
incremental  UUT  simulator  usage  costs.  There  was  considerable  difficulty 
in  coming  up  with  these  estimates.  First,  the  development  activity 
required  to  develop  a  TPS  varies  widely,  depending  on  the  complexity  of 
the  UUT.  Second,  there  is  no  consistent  information  on  costs  of  UUT 
development  in  the  open  literature. 

To  get  some  comparative  numbers  for  the  UUT  simulator  solution 
alternatives,  the  following  assumptions  and  estimates  were  made: 

•  A  nine- man- month  TPS  development  size  was  assumed. 

•  It  was  assumed  that  some  UUT  associated  development  costs 
are  incurred  to  make  the  simulator  operational.  These  costs 
include  both  entering  the  UUT  description  ard  preparing 
auxiliary  simulators  (such  as  for  a  UUT  adapter  or  special 
instrument). 

•  A  rough  estimate  was  made  for  each  of  the  simulation  levels 
and  interface  methods,  using  a  figure  of  $7K  per  man-month  to 
translate  the  cost  into  dollars. 
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TABLE  23.  INCREMENTAL  USAGE  COST  FACTORS 


•  Usage  costs  of  the  UUT  simulator  were  estimated,  based  on 

a  percentage  of  time  the  UUT  simulator  would  be  tied  up  and  on 
the  amount  of  equipment  tied  up  in  performing  the  simulations. 

A  basic  charge  of  $45  per  hour  of  UUT  simulator  CPU  time 
was  assumed. 

•  ATE  system  usage  costs  were  based  on  the  assumption  that  only 
the  CPU  portion  of  the  ATE  would  be  tied  up  for  all  solutions 
except  those  involving  the  analog  interface  method.  In  the  latter 
case  it  was  assumed  that  additional  ATE  equipment  tied  up  in 
using  the  simulator  was  about  an  order  of  magnitude  more  costly 
than  the  CPU  alone.  This  reflects  the  trend  that  CPU  costs  are 
still  going  down,  whereas  instrument  complexity  and  costs  are 
going  up. 


Figure  13  is  obtained  when  the  results  of  these  costing  exercises  are 
plotted. 

•  The  analog  interface-based  solutions  are  the  most  expensive  to 
develop  and  to  use.  Because  there  is  not  a  proportional  payoff, 
this  interface  is  not  recommended  for  use  in  a  UUT  simulator 
system. 

•  There  is  not  a  clear  cost  advantage  for  either  the  software-only 
or  the  instrument  bus  interface-based  solutions.  Factors  other 
than  cost  must  be  used  to  determine  which  is  optimum  for 
specific  cases. 
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Figure  13.  Summary  Costs  for  UUT  Simulator 
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The  following  are  additional  minor  conclusions: 
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•  If  a  family  of  UUTs  can  be  adequately  served  by  one  level  of 
simulation,  a  UUT  simulator  system  based  on  this  simulation 
only  can  become  a  cost-effective  UUT  simulator  system.  The 
primary  saving  is  development  costs. 

•  A  major  part  of  the  initial  development  costs  are  independent 
of  the  simulation  technique. 


SECTION  5 


SOLUTION  DISCUSSION 


This  section  provides  more  comments  on  the  UUT  simulator.  The  following 
specific  areas  are  presented: 

•  Solution  Recommendations  and  Discussions:  More  details  of 
the  recommendations  are  presented  and  implications  are 
discussed. 

•  Usage  Scenarios:  A  few  example  scenarios  of  how  the  UUT 
simulator  would  be  used  are  outlined. 

•  Additional  Usages:  The  UUT  simulator  can  be  used  potentially 
in  areas  other  than  just  supporting  V&V,  Some  of  these  areas 
are  described  below. 

SOLUTION  RECOMMENDATIONS  AND  COMMENTS 

Our  recommendation  is  to  ultimately  migrate  to  the  multilevel  UUT 
simulator  solution.  In  the  previous  section  it  was  seen  that  the  multilevel 
solutions  had  a  high  development  cost  associated  with  them.  If  over  a 
period  of  time  UUT  simulators  are  developed  to  use  individual  simulation 
levels  but  of  different  types,  the  cumulative  development  costs  would  not 
be  any  lower.  Unless  such  individual  developments  are  closely  coordinated, 
the  result  will  not  be  as  unified  as  if  a  coherent  approach  to  a  multilevel 
simulation  solution  were  started  early  in  the  UUT  simulator  development. 
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Perhaps  the  most  important  factor  is  the  existence  of  a  unified  approach 
from  the  start;  different  levels  of  simulation  can  then  be  brought  into  the 
system  as  they  are  needed. 


It  was  seen  in  the  previous  section  that  on  a  cost  basis  the  software-only 
and  bus  interface  solutions  are  about  equal.  The  specific  choice  between 
these  two  depends  on  the  nature  of  the  ATE  system  which  the  UUT  simulator 
has  to  support.  The  following  are  the  two  main  cases: 

•  If  the  ATE  system  is  strongly  based  on  a  standard  ATLAS 
compiler  and  if  it  can  be  guaranteed  that  no  "idiosyncracies" 
of  the  compiler  or  ATE  system  get  in  the  way  of  having  a 
standard,  reliable,  and  controlled  interface  to  the  UUT  simulator, 
the  software-only  solution  is  good. 

•  If  the  ATE  system  is  based  on  a  well-defined  bus  structure 
connecting  the  Central  Processor  of  the  ATE  system  to  the 
stimuli  generation  and  measurement  instruments,  the  UUT 
simulator  may  be  best  interfaced  to  this  bus.  The  clear  advantage 
is  that  the  simulator  interface  does  not  depend  on  the  source 
language  that  the  test  program  is  written  in.  For  example,  ATLAS 
may  be  the  required  standard,  but  to  improve  performance  of  a 
specific  test,  a  TPS  developer  may  request  and  receive  a  waiver 
to  use  another  language  in  a  minor  part  of  the  system.  Such  a 
waiver  would  have  a  severe  negative  impact  on  the  software-only 
solution.  The  bus  interface  would  still  be  valid. 

A  key  aspect  of  the  decision  concerning  which  interface  to  use  is  how 
well-defined  the  interface  is  and  how  well  it  can  be  controlled  by  the  UUT 
simulator  developer. 


Another  factor  in  the  software-only  vs  bus  interface  decision  is  the 
relative  timing  of  the  interface  definition  and  its  implementation.  Because 
it  is  desirable  to  start  definition  of  the  UUT  simulator  early,  the  choice 
may  have  to  depend  on  which  of  the  two  is  available  first.  For  example, 
if  the  software  interface  can  be  well-defined  for  the  MATE  program 
early,  then  the  software-only  solution  should  perhaps  be  favored. 

Like  other  new  technologies,  the  use  of  a  UUT  simulator  does  imply  some 
risk  factors,  particularly  during  the  early  usages.  The  following  are 
some  cautions  to  be  observed  to  prevent  less  than  desirable  early  usage 
experiences  from  preventing  achievement  of  benefits  when  concept  is  more 
mature: 

•  A  UUT  simulator  learning  curve  will  apply  during  the  early 
UUTs  simulated  on  a  generic  UUT  simulator.  Because  start-up 
and  training  problems  may  be  encountered  during  the  implementa¬ 
tion  of  the  early  UUT  simulators,  the  apparent  implementation 
time  may  appear  too  long  or  the  net  cost-effectiveness  may 
appear  poor. 

•  TPS  development  costs  may  actually  appear  higher  than  without 
the  use  of  the  UUT  simulator.  As  seen  previously,  some 
reasons  are  the  cost  of  developing,  modifying,  and  maintaining 
the  simulator.  In  addition,  the  existence  of  the  simulator  will 
provide  deeper  insights  into  what  the  test  programs  are  and  are 
not  doing.  The  process  of  resolving  these  differences  will  take 
more  time  than  without  the  simulator.  There  is  also  the  possibility 
that  errors  in  the  UUT  simulator  will  cause  some  such  problems; 
these  would  have  to  be  treated  like  the  design  problems  in  the  UUT 
itself. 


In  order  to  provide  answers  to  the  questions  and  issues  raised  above  for 
the  MATE  program,  a  UUT  simulator  demonstration  program  should  be 
initiated.  The  following  are  specific  justifications: 

•  For  reasons  other  than  development  costs,  the  concept  of  a 

UUT  simulator  needs  to  be  pursued.  The  following  are  some 

of  these  reasons: 

—  Unavailability  of  UUTs.  If  a  UUT  is  not  available  in  a 
development  that  is  on  the  critical  path  of  a  weapons  program, 
there  may  be  cost  impacts  outside  of  TPS  development.  A 
UUT  simulator  can  help  reduce  this  problem. 

—  Long  term  payoffs:  Reduced  LRU  downtime,  reduced  main¬ 
tenance  time  and  ultimately  reduced  testing  time  will  have 
larger  system  cost  benefits  than  cost  figures  associated  with 
just  the  TPS  development. 

—  Schedule  compression  of  TPS  development:  Once  the  UUT 
simulator  concept  is  mature  and  supported  at  an  adequate 
level,  the  UUT  simulator  will  help  to  reduce  the  amount  of 
time  to  develop  a  TPS  and  verify  its  correctness. 

—  Maintenance  advantages:  There  are  additional  advantages  which 
can  be  realized  later  during  the  maintenance  period  of  a  TPS. 

A  UUT  simulator  can  provide  support  for  revalidation  of  a 
TPS  either  to  correct  a  TPS  problem  or  to  accommodate  an 
engineering  change  in  the  UUT  itself,  which  force  a  change  in 
the  test  program.  A  UUT  simulator,  if  maintained  during  the 
lifetime  of  the  TPS  will  also  support  the  resolution  of  unanticipated 
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problems  in  the  field  by  supporting  contingency  analyses. 

Analysis  of  what  happened  in  catastrophic  failures  may  be 
another  benefit. 

•  The  cost-effectiveness  of  a  generic  UUT  simulator  is  not  yet 
clear.  The  demonstration  program  can  address  some  of  the 
key  questions  on  the  cost  implications  to  implement  and  maintain 
additional  UUT  simulators.  Such  experiments  could  be  run 
after  a  demonstration  UUT  simulator  system  is  available. 

•  The  relationship  of  a  UUT  simulator  to  Automatic  Test  Program 
Generation  is  not  clear.  The  indications  are  that  the  UUT  simulator 
can  be  complementary  to  ATPG,  covering  V&V  support  in  those 
areas  where  test  components  must  still  be  generated  manually. 

USAGE  SCENARIOS 

The  purpose  of  this  section  is  to  present  some  sample  scenarios  in  which 
a  UUT  simulator  is  used.  The  following  scenarios  are  described: 

1.  Minimal  Baseline  Scenario 

2.  Interactive  Scenario 

3.  Batch  Mode  Scenario 


4.  Model  Changing  Scenario 
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Minimal  Baseline  Scenario 

•  Prior  to  the  development  of  the  specific  UUT  simulator,  it  is 
assumed  that  a  generic  simulator  that  supports  the  required 
functionality  is  available. 

•  The  UUT  is  assumed  already  developed. 

•  The  development  of  the  specific  UUT  simulator  is  assumed 
started  prior  to  the  start  of  the  ATLAS  test  program  development 
so  that  it  is  available  when  test  program  V&V  starts. 

•  First  the  initial  version  of  the  UUT  simulator  is  built  and 
thoroughly  checked  for  consistency. 

•  The  developed  ATLAS  test  programs  are  then  played  against  the 
simulator  with  careful  control  over  the  execution. 

—  Single  step  runs  are  used  to  check  on  interface  activity  and 
test  program  actions. 

—  Recovery  points  are  retained  to  facilitate  restarts  after 
test  program  errors  are  encountered. 

—  Detected  errors  in  the  test  programs  are  incrementally 
corrected  and  re-verified. 

•  A  small  set  of  simulated  faults  are  selected  and  built  incrementally 
into  the  UUT  simulator  model. 

•  Recovery  is  made  at  a  point  before  a  simulated  fault  is  to  be 
exercised  in  both  the  UUT  simulator  and  in  the  ATE  system. 

•  Fault  simulation  is  activated,  and  the  action  of  the  ATLAS  test 
program  and  the  UUT  simulator  are  recorded  and  later  analyzed 
for  proper  operation. 
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•  Further  faults  are  activated,  one  at  a  time,  to  check  test  program 
logic  and  to  verify  proper  operator  messages  and  expected 
operator  actions. 

•  This  process  is  repeated  by  incrementally  refining  and  increasing 
the  fault  modeling  in  the  UUT  simulator  and  by  playing  further 
tests  against  the  simulator. 

•  The  procedures  and  UUT  simulator  parameter  selections  used  to 
force  specific  faults  are  also  stored  for  later  reenactment  of 
the  executions. 

•  When  the  performance  of  the  test  program  is  adequate,  via  use 
of  the  UUT  simulator,  verification  is  continued  on  the  real  UUT, 
using  the  real  UUT  Interface  Adapter. 

•  Discrepancies  in  the  operation  of  the  test  program  against  the 
real  UUT  as  compared  to  the  UUT  simulator  are  analyzed.  Those 
discrepancies  requiring  only  simple  changes  in  the  UUT  simulator 
are  used  to  update  the  simulator  for  future  revalidation  of  the 
test  program  after  changes  have  been  made.  Those  discrepancies 
which  imply  extensive  changes  in  the  UUT  simulator  need  to  be 
evaluated  for  cost-effectiveness.  It  may  be  better  to  leave  the 
simulator  deficient  and  require  that  the  specific  test  be  revalidated 
in  the  future  on  a  real  UUT. 

Interactive  Scenario 


•  Assume  that  the  baseline  scenario  holds.  The  interactive  scenario 
is  an  example  of  a  detailed  interactive  activity  in  the  middle  of 
the  baseline  scenario. 
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•  Suppose  that  an  inconsistency  is  discovered  in  a  signal  applied 

at  some  UUT  pins  versus  what  the  UUT  simulator  has  programmed 
into  it  as  being  acceptable. 

•  The  simulation  is  temporarily  suspended  by  the  developer. 

•  History  of  the  simulation  steps  leading  to  the  condition  is  printed 
on-line.  This  history  would  include  activities  in  the  interface 
between  the  ATE  and  the  interface  adapter  and  between  the 
interface  adapter  and  the  UUT. 

•  The  developer  examines  the  test  program  and  the  corresponding 
trace  output. 

•  If  the  information  is  not  enough,  recovery  is  made  to  a  previous 
restart  point,  and  more  detailed  tracing  is  requested. 

•  When  found,  the  offending  program  step  is  corrected  and  the 
test  program  is  re- compiled. 

•  A  recovery  point  is  selected  and  the  test  program  execution  is 
restarted. 

•  Trace  levels  are  set  high  to  provide  output  to  ensure  that  the 
correction  was  made  properly. 

•  If  correct,  the  change  to  the  source  is  inserted  permanently. 

•  The  session  is  continued. 
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Batch  Mode  Scenario 


•  Assume  that  the  baseline  scenario  holds.  This  scenario  is  an 
example  of  how  the  iterative  planning,  preparation,  testing,  and 
analysis  would  be  done  under  the  constraints  of  a  batch  mode 
operation  of  the  UUT  simulator. 

•  The  UUT  simulator  is  assumed  entered,  and  the  ATLAS  test 
program  segments  to  be  verified  have  been  designed  and  coded. 

•  The  steps  to  be  verified  are  selected  and  batched  into  independent 
groups. 

•  The  faults  to  be  exercised  are  selected,  and  a  plan  for  the  batch 
runs  is  made. 

•  Trace  levels  are  preplanned  to  get  the  proper  amount  of  infor¬ 
mation  from  the  simulation  runs. 

•  The  input  is  prepared  for  one  batch  run  and  submitted. 

•  While  waiting  for  outputs,  inputs  for  testing  other  batch  groups 
are  prepared,  and  results  of  previous  runs  are  analyzed. 

•  When  the  results  from  the  batch  run  are  back,  they  are  analyzed 
for  expected  responses  and  for  errors. 

•  If  no  errors  appear,  further  test  groups  are  prepared. 

•  If  there  are  errors,  reasons  for  the  errors  are  analyzed. 

•  If  the  reasons  cannot  be  found  after  a  reasonable  analysis  time, 
the  batch  is  submitted  for  rerun,  but  with  more  detailed  traces 
to  support  isolation  of  the  cause  of  the  error. 
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•  The  above  cycle  is  repeated  continuously  with  two  to  three 
batch  groups  in  process  concurrently  to  prevent  idle  time 
while  waiting  for  output  from  the  batch  runs. 

Model  Changing  Scenario 

•  Assume  that  the  UUT  simulator  has  multiple  levels  of  simulation. 

•  Assume  a  normal  session  has  been  started. 

•  Suppose  that  a  specific  response  from  the  simulator  doesn't 
look  good  enough;  for  example,  a  better  output  waveform  is 
needed. 

•  The  session  is  stopped  and  set  to  restart  at  a  previous  recovery 

point.  •  *  *  - 

•  Interactive  control  over  the  simulator  is  exercised  to  select 
a  more  detailed  model  for  the  required  section  of  the  UUT. 

•  Simulation  is  continued,  and  the  output  is  monitored  carefully 

to  insure  that  the  improved  response  has  been  realized.  However, 
execution  is  probably  slower. 

•  If  results  are  OK,  the  procedural  steps  to  activate  the  more 

detailed  simulation  are  recorded  for  replay  during  future 

revalidations  of  the  same  section  of  test  program. 

% 

•  The  model  is  restored  to  the  less  detailed  but  faster  version, 
and  the  simulation  session  is  continued. 
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ADDITIONAL  USAGES 


During  the  conduct  of  this  study  most  of  the  emphasis  was  placed  on 
having  the  UUT  simulator  support  V&V  and  TPS  development  activities. 
Given  that  one  has  a  UUT  simulator  available,  there  is  a  potential  set 
of  additional  uses  aside  from  the  mainstream  use.  The  discussion  below 
addresses  some  of  these  potential  uses. 

Semiautomated  Support  for  Test  Program  Generation 


While  discussing  the  relationship  of  a  UUT  simulator  to  ATPG,  we 
determined  that  a  very  good  use  of  a  circuit  level  form  of  simulator  would 
be  to  actually  support  the  semiautomated  generation  of  test  programs. 

The  developer  would  perform  the  analysis  to  determine  what  stimuli  should 
be  applied,  cause  them  to  be  applied  to  the  UUT  simulator,  and  watch  the 
response.  The  response  could  also  be  captured  and  later  used  for 
signature  recognition.  Such  a  use  of  a  UUT  simulator  would  be  comple¬ 
mentary  to  some  forms  of  analog  ATPG  and  is  useful  when  A  ATPG  is 
not  yet  available  or  determined  for  a  specific  case  not  to  be  cost- 
effective. 

Training  Support 


ATE  maintenance  and  operator  training  are  a  large  area  of  concern  in 
the  Air  Force.  A  UUT  simulator  would  be  capable  of  supporting  some  of 
the  more  advanced  aspects  of  maintenance  and  operator  training.  Some 
forms  of  training  in  which  the  exact  operator  or  technician  interface  is 
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required  (i.  e. ,  realistic  instrument  panels  and  controls)  would  not  be 
appropriate  for  use  with  a  UUT  simulator  as  required  for  V&V.  However, 
some  of  the  more  advanced  training  in  which  the  student  can  be  expected 
to  interpret  results  from  a  CRT  console  are  potentially  supportable.  For 
specific  training  exercises  in  which  the  UUT  simulator  would  only  be 
required  to  keep  the  ATE  system  cycling  (i.  e. ,  all  student  interaction 
is  with  the  ATE  system),  the  UUT  simulator  could  also  be  useful. 

UUT  Specification  in  TRD 

There  is  currently  some  movement  toward  the  use  of  ATLAS  as  a  means 
of  specifying  test  requirements  more  precisely  in  TRDs.  One  step 
beyond  this  philosophy  is  to  use  a  UUT  simulator  that  is  defined  in  a 
concise  and  unambiguous  language  along  with  a  precise  description  of 
the  tests  to  be  performed.  The  concept  of  a  UUT  simulator  as  well  as 
much  support  would  have  to  be  matured  to  a  high  degree  before  this  is 
possible. 

Integrated  Repository  of  UUT  Behavior  in  Field 

The  UUT  simulator  can  be  used  as  a  continually  updated  storehouse  of 
information  on  a  specific  UUT.  As  failures  as  well  as  good  experiences 
are  obtained  from  the  field,  the  UUT  simulator  can  be  updated.  The 
resulting  accumulated  information  can  lager  be  used  as  a  basis  for 
decisions  on  LRU  redesign  or  on  rework  of  tests  to  correct  for  poor 
field  performance. 
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Support  for  Post  Mortem  of  Catastrophic  Failures 


If  catastrophic  failures  occur  because  of  factors  which  were  not  even 
entered  into  the  design  of  the  LRU,  the  UUT  simulator  may  be  used  as 
a  means  of  postulating  what  might  have  happened  and  what  could  be  done 
to  prevent  a  recurrence  of  the  failure. 

Interface  Adapter  Verification 


Adapter  verification  consists  of  verification  of  the  following: 

•  Continuity  from  the  ATE  to  the  desired  UUT  test  point 

•  Isolation  between  UUT  test  points 

•  Insulation  breakdown  voltage  measurements 

•  Functionality  of  adapter  active  circuitry  or  fixtures 

•  Transmission  line  characteristics 

•  Noise  immunity 

Verification  of  transmission  line  parameters  and  noise  immunity  is 
done  only  when  problems  of  this  type  are  anticipated  or  are  actually 
experienced.  This  puts  these  considerations  in  the  category  of  exceptional 
tests,  generally  requiring  special  equipment  and  fixturing.  Insulation  and 
Interface  Adapter  (IA)  voltage  breakdown  measurements  often  require 
specially  designed  power  supplies  and  loading  fixtures.  It  is  very  difficult 
to  generalize  about  verification  of  the  functionality  of  interface  adapter 
active  circuitry.  IA  active  circuitry  may  serve  to  eliminate  costly 
ATE  modification,  improve  noise  immunity,  and  capture  very  high 
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frequency  low  duty  cycle  signals  or  other  signal  conditioning  functions.  The 
IA  fixtures  range  in  spectrum  from  simple  supports  to  complex  optic 
benches  and  environmental  simulators  (such  as  acoustic  or  complex 
electromagnetic  environments).  This  uniqueness  implies  a  requirement 
for  unique  operational  verification  procedures  beyond  the  envisioned 
scope  of  the  UUT  simulator. 

The  remaining  two  subtasks,  verification  of  continuity  from  the  ATE  to 
a  UUT  testpoint  and  isolation  between  test  points,  appear  to  be  addressable 
by  the  UUT  simulator.  The  results  of  our  survey  indicate  that  miswiring 
and  poor  workmanship  were  the  most  common  IA  problems.  The  debug 
of  the  IA  appears  to  be  a  profitable  area  to  devote  some  effort. 

Anticipated  problem  areas  in  adapting  the  UUT  simulator  to  this  task 
are  as  follows: 

•  The  variety  of  physical  configurations  used  for  present 
ATE/UUT  IAs. 

Configuration  differences  range  from  different  sized  and 
interlocking  connectors  to  conceptual  differences  such  as 
octopus  types  of  cables. 

•  The  validity  of  the  IA  description  to  the  UUT  simulator. 

Although  this  problem  may  appear  trite,  it  is  believed  that 
many  adapter  errors  are  due  to  the  menial  nature  of  the  tasks 
of  describing  and  building  IAs. 
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Because  the  results  of  the  survey  indicate  that  validation  puts  heavy 
demands  on  ATE  station  time,  it  would  appear  wise  to  remove  the  IA 
verification  task  from  the  ATE  if  possible.  Assuming,  then,  that  an 
ATE  interface  is  common  to  many  UUTs,  the  problem  of  the  variety 
of  physical  configurations  can  be  addressed  by  building  a  UUT  simulator 
to  ATE  interface  adapter  which  then  becomes  common  to  all  UUTs  tested 
on  that  ATE  (assuming  that  the  UUT  simulator  consists  of  a  set  of 
hardware  and  software  separate  from  the  ATE). 


* . 


2 


T 


SECTION  6 


CONCLUSIONS 

The  following  conclusions  summarize  our  findings  during  the  conduct  of 
the  UUT  Simulator  Feasibility  Investigation  Study  program: 

•  The  feasibility  of  simulators  for  classes  of  UUTs  in  the  digital, 
analog,  and  hybrid  area  is  clear;  their  cost-effectiveness  in 
the  support  of  V&V  is  still  to  be  validated. 

There  is  an  abundance  of  all  levels  of  simulation  techniques 
which  are  capable  of  being  used  for  UUT  simulations.  The 
issues  are  related  more  to  cost-effectiveness  than  feasibility. 

The  computer  systems  technology  which  is  available  for  realizing 
these  simulations  is  becoming  less  expensive,  implying  a 
greater  potential  for  UUT  simulator  practicality. 

There  may  be  some  UUTs  which  are  not  feasible  to  simulate 
fully;  this  fact  has  been  recognized  in  the  study  and  acknowledged 
through  statements  that  the  real  UUT  cannot  be  replaced  fully 
with  a  UUT  simulator. 

•  To  be  most  effective,  the  UUT  simulator  development  must 
start  early  in  the  development  cycle  of  an  LRU,  and  the  simulator 
must  be  based  on  the  LRU  design  or  schematics. 
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The  initial  emphasis  in  our  study  was  oriented  toward  minimizing 
test  program  development  costs  and  V&V  costs.  Although  this 
goal  in  itself  is  a  valid  one,  direction  from  the  Air  Force  indicated 
that  the  UUT  simulator  should  be  oriented  more  toward  ensuring 
that  the  V&V  process  provides  better  checks  on  the  TRD  in 
addition  to  the  test  programs.  The  rationale  is  that  if  higher 
quality  test  programs  going  to  the  field  result  from  the  improved 
V&V,  higher  effective  payoff  will  be  realized  by  the  Air  Force. 

The  validation  of  such  a  payoff  is  not  supportable  now  with 
available  data  and  would  require  monitoring  over  a  number  of 
years  to  be  quantifiable. 

The  UUT  simulator  should  be  used  as  an  orthogonal  check  on  the 
validity  and  completeness  of  the  test  programs.  Assessment  of 
the  degree  of  completeness  of  fault  detection  and  isolation  of 
the  test  programs  to  minimize  LRU  down-time  is  a  key  payoff 
factor  for  the  Air  Force.  This  orthogonal  check  is  realized  if 
the  simulator  is  developed  independent  of  the  team  developing 
the  test  programs. 

Another  check  is  afforded  if  the  UUT  simulator  development  and 
independent  V&V  teams  view  the  UUT  from  a  completely 
different  perspective  than  the  test  program  developers.  When 
these  two  views  are  completely  reconciled  by  having  the  test 
programs  exercising  the  UUT  simulator,  a  high  degree  of 
confidence  in  the  test  programs  can  be  achieved. 
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•  The  UUT  simulator  should  be  based  on  multiple  levels  of 

simulation  and  on  either  a  software-only  or  instrument  bus  inter¬ 
face  with  the  ATE  system  on  which  the  test  programs  are  executed. 
Solutions  based  on  analog  signal  interfaces  are  not  recommended. 

It  was  determined  that  the  analog  signal  interface  solutions  were 
not  only  costly  to  develop  and  use  but  also  less  desirable  from 
a  utility  point  of  view.  Such  an  interface  may  introduce  extraneous 
problems  into  the  V&V  process,  which  may  detract  from  the 
usefulness  of  the  simulator.  Comments  resulting  from  the  survey 
that  was  conducted  also  indicated  low  credibility  in  the  cost- 
effectiveness  of  such  solutions. 

From  the  available  data,  it  was  not  possible  to  determine  whether 
the  software-only  solution  {i.  e. ,  the  simulator  running  in  the 
same  computer  as  the  test  program  interpreter)  was  more  cost- 
effective  than  the  solutions  using  an  instrument  bus  interface  to 
the  ATE  system.  The  choice  must  depend  on  the  relative 
degree  of  standardization  on  the  test  programming  language 
(ATLAS)  and  the  degree  of  standardization  on  the  bus  interface 
used  in  the  ATE  system. 

The  multi-level  simulation  recommendation  stems  from  the  fact 
that  different  levels  of  simulation  can  be  used  in  an  optimum 
manner,  resulting  in  the  most  cost-effective  usage  in  the 
long  run.  For  specific  uses,  specific  levels  of  simulation  may 
be  more  cost-effective. 
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In  using  the  UUT  simulator  as  recommended  above,  the  TPS 
development  cost  may  actually  increase,  while  decreasing  the 
maintenance  life-cycle  costs. 

The  first  reason  for  this  potential  increase  is  that  the  UUT 
simulator  must  also  be  developed.  The  second  reason  is  that 
the  orthogonal  checks  provided  by  the  UUT  simulator  will  find 
more  bugs  during  the  TPS  development  cycle,  which  may  other¬ 
wise  have  slipped  through  acceptance  testing.  Therefore  the 
"sell-off"  of  the  TPS  may  be  delayed.  But  the  resulting 
increase  in  test  program  quality  should  more  than  offset  the 
development  cost  increase. 

A  UUT  simulator  is  not  a  total  or  the  only  solution  in  supporting 
.V&V  and  in  decreasing  test  program  development  and  maintenance 
costs. 

The  simulator  is  not  a  total  solution  because  the  recommended 
forms  of  simulation  leave  some,  even  though  ultimately  small, 
part  of  the  validation  uncovered.  For  those  checks  the  real  UUT 
is  still  needed. 

There  are  other  technologies,  such  as  automatic  test  program 
generation,  which  if  satisfactorily  realized,  can  be  more 
cost-effective  in  reducing  test  program  development  and 
maintenance  costs  than  the  use  of  a  UUT  simulator.  For 
example,  in  those  areas  where  digital  ATPG  applies,  a  UUT 
simulator  is  not  needed,  except,  perhaps,  to  validate  that  the 
ATPG  generates  correct  results. 
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A  UUT  simulator  is  most  effective  where  ATPG  does  not  apply, 
because  circuit  complexity  is  higher  than  that  handled  by  the 
ATPG  or  those  areas  where  ATPG  has  not  yet  been  developed, 
or  it  is  not  cost-effective. 

•  Test  program  developers  and  validators  must  be  convinced  of  the 
merits  of  a  UUT  simulator.  A  moderate  amount  of  skepticism 
over  the  cost-effectiveness  of  a  UUT  simulator  was  encountered 
while  conducting  the  survey.  A  demonstration  program  addressing 
further  issues  and  supporting  the  validation  of  the  UUT  simulator 
concept  is  necessary  before  large  scale  usage  of  UUT  simulators 
can  be  started. 

•  Additional  UUT  simulator  cost-effectiveness  issues  must  be 
resolved.  At  this  point  the  nature  of  the  ultimate  recommendation 
on  the  use  of  a  UUT  simulator  in  the  MATE  context  is  not  known; 
partly  because  the  outputs  of  the  MATE  Program  are  not  yet 
known.  It  appears  that  any  one  of  the  following  is  still  a 
possible  outcome  of  the  UUT  simulator  demonstration  program: 

— The  recommendation  to  develop  a  generic  UUT  simulator 

system,  analogous  to  an  ATE  system,  which  is  modular  and 
can  be  tailored  to  quickly  and  cost-effectively  realize  specific 
UUT  simulators. 

—  The  recommendation  to  develop  a  family  of  less  generic  but 
lower  cost  UUT  simulator  systems  that  would  be  oriented  to 
specific  classes  of  UUTs. 

—  The  recommendation  to  not  construct  any  generic  system,  but 
rather  to  provide  guidelines  and  perhaps  software  modules  to 
support  developing  unique  UUT  simulators  based  on  specific 
weapon  program  requirements. 
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CLASSIFICATIONS  FOR  TRDs 


245 


Example  1: 


Stimulus  -  level  2;  response  -  level  2 
RF  amp  TP  67 

•  Part  1 

A.  Stimulus 

1.  Vary  power  level  at  2JI  for  band  1  and  band  2. 

a.  Band  1  is  (F2  +  F5)/2. 

b.  Band  2  is  (F9  +  F4)/2. 

2.  Vary  CW  inputs  from  -17.  5  dBm  to  -6.  5  dBm  in  1. 0  dBm 
increments. 

B.  Response 

1.  Measure  the  RF  power  at  each  increment. 

a.  Band  1,  power  is  PIC  min. 

b.  Band  2,  power  is  P2C  min. 

Example  2: 

Stimulus  -  level  1;  response  -  level  8 
Fault  locator,  TP  32 

A.  Apply  a  1  usee  negative  going  pulse  to  connections  1A3P2-5 
and  1 A3P2-26. 

B.  The  UTT,  analyzer  strobe,  and  ROM  clock  will  have  the 
following  characteristics: 
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Stimulus  -  level  5;  response  -  level  3 
Fault  locator,  TP  24 

•  Part  1 

A.  Stimulus 

1.  Apply  26,3  VDC  for  10  msec  to  the  connections  1A3P1-16 
and  1 A3P1-15. 

2,  Reduce  the  5  VDC  standby  voltage  to  4.45  VDC  within  100 
usee  at  the  connections  1A3P1-13  and  1A3P1-35. 

B.  Response 

1,  Test  signal  goes  off  anytime  the  voltage  at  1A3P1-13  and 
1 A3P1-35  is  between  4.  45  and  4.  55. 
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•  Part  2 


A.  Stimulus 

Apply  5  VDC  to  connections  1A3P1-13  and  1A3P1-35. 

2.  Apply  26.3  VDC  for  10  msec  to  the  connection  1A3P1-16 
and  1A3P1-15. 

3.  Increase  the  5  VDC  standby  voltage  to  5.  55  VDC  within 
100  usee  at  the  connections  1A3P1-13  and  1A3P1-35. 

B.  Response 

1.  Test  signal  goes  off  anytime  the  voltage  at  1A3P1-13  and 
1 A3P1  -35  is  between  5.  45  and  5.  55  VDC. 

Example  4: 

Stimulus  -  level  1;  response  -  level  1 
Fault  locator,  TP  18 

•  Part  1 

A.  Stimulus 

1.  Ground  connections  1A3P1-67  and  1A3P1-35. 

B.  Response 

1.  Wait  200  msec,  then  perform  test. 

2.  5  ohms  max.,  1A3P1-69  and  1 A3P1-35 
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•  Part  2 


A.  Stimulus 

1.  Open  connections  1A3P1-67  and  1A3P1-35. 

B.  Response 

1.  Wait  200  msec,  then  perform  test. 

2.  100K  ohms  min.,  1A3P1-69  and  1A3P1-35 

Example  5: 

Stimulus  -  level  2;  response  -  level  1 
Fault  locator,  TP  38 

•  Part  1 

A.  Stimulus 

1.  Apply  a  1  usee  negative  going  pulse  (logic  0)  to  the 
connection  1A3P2-5  and  1A3P2-26. 

B.  Response 

1.  Test  signal  is  on,  logic  0  for  connections  1A3P2-6  and 
1 A3P2-47. 

•  Part  2 

A.  Stimulus 

1.  Apply  a  1  usee  negative  going  pulse  (logic  0)  to  the 
connections  1A3P2-4  and  1A3P2-25. 

B.  Response 

1.  The  test  signal  is  off,  logic  1  for  connections  1A3P2-6 
and  1 A3P2-47. 
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Example  6: 


Stimulus  -  simple;  response  -  level  6 
Fault  locator,  TP  41 

A.  Stimulus 

1.  Apply  26.3  VDC  to  connections  1A3P1-16  and  1A3P1-15. 

B.  Response 

1.  Test  is  performed  at  UUT  timing  rate. 

2. *  B20  of  all  512  words  are  tested. 

a.  Enable  -  logic  1  is  0.  4  VDC  max. 

b.  Inhibit  -  logic  0  is  2.4  VDC  min. 

3.  Data  must  be  sampled  and  read  within  6  to  8  usee  after 
ROM  clock  pulse. 

Example  7: 


Stimulus  -  level  2;  response  -  level  3 
RF  amp  TP  85 

•  Part  1 

A.  Stimulus 

1.  Apply  0.3  VDC  at  connections  2J2-59  and  2J2-60. 

2.  Apply  a  100  Hz  square  wave  with  50%  duty  cycle  and 
amplitude  0.3  VDC  at  connections  2J2-52  and  2J2-51. 

3.  Apply  CW  RF  at  frequencies  (F5  +  F2)/2  (band  1)  and 
(F9  +  F4)/2  (band  2). 
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a.  Apply  -14.  7  dBm  at  band  1. 

b.  Apply  -17.4  dBm  at  band  2. 

B.  Response 

1.  Check  depth  of  modulation  at  connections  2J5,  2J6-P, 
and  2J6-S. 

a.  For  band  1,  the  depth  of  modulation  is  13  dB  ±  1.8  dB 
below  PIC. 

b.  For  band  2,  the  depth  of  modulation  is  13  dB  ±  1. 8  dB 
below  P2C. 

•  •  •  •  •  •  « 

•  Part  2 

A.  Stimulus 

1.  Repeat  part  1  stimulus  using  frequencies  F2  (band  1) 
and  F4  (band  2). 

B.  Response 

1.  Repeat  part  1  response  using  frequencies  F2  (band  1) 
and  F4  (band  2). 

•  Part  3 

A.  Stimulus 

1.  Repeat  part  1  stimulus  using  frequencies  F5  (band  1) 
and  F9  (band  2), 

B.  Repeat  part  1  response  using  frequencies  F5  (band  1)  and 
F9  (band  2). 
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Example  8: 


APPENDIX  C 


ATLAS  TEST  PROGRAM  FLOW  DIAGRAM  EXAMPLES 
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